Aperture scanning fourier ptychographic imaging

ABSTRACT

Certain aspects pertain to aperture-scanning Fourier ptychographic imaging devices comprising an aperture scanner that can generate an aperture at different locations at an intermediate plane of an optical arrangement, and a detector that can acquire lower resolution intensity images for different aperture locations, and wherein a higher resolution complex image may be constructed by iteratively updating regions in Fourier space with the acquired lower resolution images.

CROSS-REFERENCES TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.14/448,850, titled “Aperture Scanning Fourier Ptychographic Imaging” andfiled on Jul. 31, 2014, which claims benefit of U.S. Provisional PatentApplication No. 61/860,786, titled “Generalized Ptychographic Imagingwith Optical Transfer Function Modulation” and filed on Jul. 31, 2013and U.S. Provisional Patent Application No. 61/868,967, titled“Alternative Optical Implementations for Fourier PtychographicMicroscopy” and filed on Aug. 22, 2013; all of these applications arehereby incorporated by reference in their entireties and for allpurposes.

BACKGROUND OF THE INVENTION

Certain embodiments described herein generally relate to imagingtechniques, and more specifically to methods, devices, and systems forFourier ptychographic imaging.

Imaging lenses ranging from microscope objectives to satellite-basedcameras are physically limited in the total number of features they canresolve. These limitations are a function of the point-spread function(PSF) size of the imaging system and the inherent aberrations across itsimage plane field of view (FOV). Referred to as the space-bandwidthproduct, the physical limitation scales with the dimensions of the lensbut is usually on the order of 10 megapixels regardless of themagnification factor or numerical aperture (NA). A discussion ofspace-bandwidth product of conventional imaging systems can be found inLohmann, A. W., Dorsch, R. G., Mendlovic, D., Zalevsky, Z. & Ferreira,C., “Space-bandwidth product of optical signals and systems,” J. Opt.Soc. Am. A. 13, pages 470-473 (1996), which is hereby incorporated byreference for this discussion. While conventional imaging systems may beable to resolve up to 10 megapixels, there is typically a tradeoffbetween PSF and FOV. For example, certain conventional microscopeobjectives can offer a sharp PSF (e.g., 0.5 μm) across a narrow FOV(e.g., 1 mm), while others imaging systems with wide-angle lenses canoffer a wide FOV (e.g., 10 mm) at the expense of a blurry PSF (e.g., 5μm).

Certain interferometric synthetic aperture techniques that try toincrease spatial-bandwidth product are described in Di, J. et al., “Highresolution digital holographic microscopy with a wide field of viewbased on a synthetic aperture technique and use of linear CCD scanning,”Appl. Opt. 47, pp. 5654-5659 (2008); Hillman, T. R., Gutzler, T.,Alexandrov, S. A., and Sampson, D. D., “High-resolution, wide-fieldobject reconstruction with synthetic aperture Fourier holographicoptical microscopy,” Opt. Express 17, pp. 7873-7892 (2009); Granero, L.,Mico, V., Zalevsky, Z., and Garcia, J., “Synthetic aperturesuperresolved microscopy in digital lensless Fourier holography by timeand angular multiplexing of the object information,” Appl. Opt. 49, pp.845-857 (2010); Kim, M. et al., “High-speed synthetic aperturemicroscopy for live cell imaging,” Opt. Lett. 36, pp. 148-150 (2011);Turpin, T., Gesell, L., Lapides, J., and Price, C., “Theory of thesynthetic aperture microscope,” pp. 230-240; Schwarz, C. J., Kuznetsova,Y., and Brueck, S., “Imaging interferometric microscopy,” Optics letters28, pp. 1424-1426 (2003); Feng, P., Wen, X., and Lu, R.,“Long-working-distance synthetic aperture Fresnel off-axis digitalholography,” Optics Express 17, pp. 5473-5480 (2009); Mico, V.,Zalevsky, Z., Garcia-Martinez, P., and Garcia, J., “Synthetic aperturesuperresolution with multiple off-axis holograms,” JOSA A 23, pp.3162-3170 (2006); Yuan, C., Zhai, H., and Liu, H., “Angular multiplexingin pulsed digital holography for aperture synthesis,” Optics Letters 33,pp. 2356-2358 (2008); Mico, V., Zalevsky, Z., and Garcia, J., “Syntheticaperture microscopy using off-axis illumination and polarizationcoding,” Optics Communications, pp. 276, 209-217 (2007); Alexandrov, S.,and Sampson, D., “Spatial information transmission beyond a system'sdiffraction limit using optical spectral encoding of the spatialfrequency,” Journal of Optics A: Pure and Applied Optics 10, 025304(2008); Tippie, A. E., Kumar, A., and Fienup, J. R., “High-resolutionsynthetic-aperture digital holography with digital phase and pupilcorrection,” Opt. Express 19, pp. 12027-12038 (2011); Gutzler, T.,Hillman, T. R., Alexandrov, S. A., and Sampson, D. D., “Coherentaperture-synthesis, wide-field, high-resolution holographic microscopyof biological tissue,” Opt. Lett. 35, pp. 1136-1138 (2010); andAlexandrov, S. A., Hillman, T. R., Gutzler, T., and Sampson, D. D.,“Synthetic aperture Fourier holographic optical microscopy,” Phil.Trans. R. Soc. Lond. A 339, pp. 521-553 (1992), all of which are herebyincorporated by reference for the discussion of attempts to increasespatial bandwidth. Most of the above-described interferometric syntheticaperture techniques include setups that record both intensity and phaseinformation using interferometric holography such as off-line holographyand phase-shifting holography. Interferometric holography has itslimitations. For example, interferometric holography recordingstypically use highly coherent light sources. As such, the constructedimages typically suffer from coherent noise sources such as specklenoise, fixed pattern noise (induced by diffraction from dust particlesand other optical imperfections in the beam path), and multipleinterferences between different optical interfaces. Thus the imagequality is typically worse than from a conventional microscope. On theother hand, using off-axis holography sacrifices spatial-bandwidthproduct (i.e., reduces total pixel number) of the image sensor. Adiscussion of certain off-axis holography methods can be found inSchnars, U. and Jüptner, W. P. O., “Digital recording and numericalreconstruction of holograms,” Measurement Science and Technology, 13,R85 (2002), which is hereby incorporated by reference for thisdiscussion. In addition, interferometric imaging techniques may subjectto uncontrollable phase fluctuations between different measurements.Hence, accurate a priori knowledge of the sample location may be neededto set a reference point in the image recovery process. Anotherlimitation is that many of these interferometric imaging systems requiremechanical scanning to rotate the sample and thus precise opticalalignments, mechanical control at a sub-micron level, and associatedmaintenances are required by these systems. In terms ofspatial-bandwidth product, these interferometric imaging systems maypresent little to no advantage as compared with a conventionalmicroscope.

Previous lensless microscopy such as digital in-line holography andcontact-imaging microscopy also present drawbacks. For example,conventional digital in-line holography does not work well withcontiguous samples and contact-imaging microscopy requires a sample tobe in close proximity to the sensor. A discussion of certain digitalin-line holography devices can be found in Denis, L., Lorenz, D.,Thiebaut, E., Fournier, C. and Trede, D., “Inline hologramreconstruction with sparsity constraints,” Opt. Lett. 34, pp. 3475-3477(2009); Xu, W., Jericho, M., Meinertzhagen, I., and Kreuzer, H.,“Digital in-line holography for biological applications,” Proc. NatlAcad. Sci. USA 98, pp. 11301-11305 (2001); and Greenbaum, A. et al.,“Increased space-bandwidth product in pixel super-resolved lensfreeon-chip microscopy,” Sci. Rep. 3, page 1717 (2013), which are herebyincorporated by reference for this discussion. A discussion of certaincontact-imaging microscopy can be found in Zheng, G., Lee, S. A.,Antebi, Y., Elowitz, M. B. and Yang, C., “The ePetri dish, an on-chipcell imaging platform based on subpixel perspective sweeping microscopy(SPSM),” Proc. Natl Acad. Sci. USA 108, pp. 16889-16894 (2011); andZheng, G., Lee, S. A., Yang, S. & Yang, C., “Sub-pixel resolvingoptofluidic microscope for on-chip cell imaging,” Lab Chip 10, pages3125-3129 (2010), which are hereby incorporated by reference for thisdiscussion.

A high spatial-bandwidth product is very desirable in microscopy forbiomedical applications such as digital pathology, haematology,phytotomy, immunohistochemistry, and neuroanatomy. For example, there isa strong need in biomedicine and neuroscience to digitally image largenumbers of histology slides for evaluation. This need has prompted thedevelopment of sophisticated mechanical scanning and lensless microscopysystems. These systems increase spatial-bandwidth product using complexmechanisms with high precision to control actuation, optical alignment,and motion tracking. These complex mechanisms tend to be expensive tofabricate and difficult to use and maintain.

BRIEF SUMMARY OF THE INVENTION

Aspects of this disclosure concern imaging techniques, and morespecifically methods, devices, and systems for Fourier ptychographicimaging.

Certain aspects pertain to aperture-scanning Fourier ptychographicimaging devices comprising optical elements, an aperture scanner thatcan generate an aperture at a plurality of locations at an intermediateplane of the optical elements, and a detector that can acquire lowerresolution intensity images for different aperture locations, andwherein a higher resolution complex image may be constructed byiteratively updating regions in Fourier space with the acquired lowerresolution images.

In some aspects, an aperture-scanning Fourier ptychographic imagingdevice comprises a first optical element configured to receive lightfrom a sample and a second optical element. The device further comprisesan aperture scanner configured to generate an aperture at a plurality ofaperture locations in an intermediate plane, the aperture configured topass incident light at the aperture from the first optical element tothe second optical element. The device further comprises a radiationdetector configured to receive light from the second optical element andto acquire a plurality of intensity images associated with differentaperture locations. The device further comprises a processor configuredto construct a complex image of the sample by iteratively updatingregions in Fourier space with the acquired intensity images.

In some aspects, a aperture-scanning Fourier ptychographic imagingmethod comprises illuminating a sample, receiving incident light at afirst optical element from the sample, generating an aperture at aplurality of locations at an intermediate plane, passing incident lightat the aperture from the first optical element to a second opticalelement. The method further comprises acquiring a plurality of intensityimages using a detector receiving light from the second optical elementand constructing a complex image of the sample by iteratively updatingregions in Fourier space with the plurality of intensity images.

These and other features are described in more detail below withreference to the associated drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates a schematic drawing of components of a Fourierptychographic imaging system with optical transfer function modulationat the intermediate plane.

FIGS. 2A and 2B are schematic drawings of components of anaperture-scanning Fourier ptychographic imaging system.

FIG. 3A is a schematic drawing of components of an aperture-scanningFourier ptychographic imaging system.

FIG. 3B is a schematic drawing of cross-sectional view of a display of aspatial light modulator that can be implemented in certainaperture-scanning Fourier ptychographic imaging systems.

FIG. 4 is a schematic drawing of components of an aperture-scanningFourier ptychographic imaging system comprising a DMD array.

FIG. 5 is a schematic drawing of components of an aperture-scanningFourier ptychographic imaging system comprising a DMD array.

FIG. 6 is a schematic drawing of a view of components of anaperture-scanning Fourier ptychographic imaging system comprising anLCOS array.

FIG. 7 is a schematic diagram of components of an aperture-scanningFourier ptychography imaging system.

FIG. 8 is a flowchart of an aperture-scanning Fourier ptychographyimaging method performed by an aperture-scanning Fourier ptychographyimaging system.

FIG. 9 is an aperture-scanning Fourier ptychography imaging method withdigital wavefront correction.

FIG. 9A is a flowchart of an example of sub-steps of one or more stepsof the method of FIG. 8.

FIG. 9B is a flowchart of another example of sub-steps of one or moresteps of the method of FIG. 8.

FIG. 10 is a flowchart of an aperture scanning Fourier ptychographicmethod with tile imaging, according to certain aspects.

FIG. 11 is a block diagram of subsystems that may be present inaperture-scanning Fourier ptychography imaging system.

DETAILED DESCRIPTION OF THE INVENTION

Embodiments of the present invention will be described below withreference to the accompanying drawings. The features illustrated in thedrawings may not be to scale.

I. Introduction

Fourier ptychography imaging implements a phase retrieval technique thatuses angular diversity to recover complex sample images. The recoveryprocess comprises alternating enforcement of known sample information inthe spatial domain and a fixed constraint in the Fourier domain. Thephase retrieval recovery may be implemented using any variant of analternating projections algorithm, a convex reformulation of theproblem, or any non-convex variant in-between. Instead of shifting asample laterally (i.e. applying translational diversity), Fourierptychography imaging uses a scanning spectrum constraint in the Fourierdomain to expand the Fourier passband beyond that of a single capturedimage to recover an improved-resolution complex sample image.

Certain variable-angle illumination Fourier ptychography imaging systemsuse a variable illumination source (e.g., an LED array) to illuminate asample being imaged from different illumination angles successively. Anoptical element such as a low numerical aperture objective lens filterslight from the sample. A radiation detector receives the filtered lightfrom the optical element and captures a intensity image of the sample ateach illumination angle. Multiple resolution images may be iterativelystitched together in the Fourier domain to recover a higher resolutionimage of the image. Details of some variable-angle illumination Fourierptychography systems, devices, and methods can be found in U.S. patentapplication Ser. No. 14/065,280, titled “Fourier Ptychographic ImagingSystems, Devices, and Methods” and filed on Oct. 28, 2013 and in U.S.patent application Ser. No. 14/065,305, titled “Fourier PtychographicX-ray Imaging Systems, Devices, and Methods,” which are herebyincorporated by reference for these details.

In some aspects, certain Fourier ptychography imaging systems describedherein comprise an aperture scanner that can generate an aperture at aplurality of N aperture locations at an intermediate plane of theoptical arrangement. For example, the aperture may be generated at aFourier plane conjugate the sample plane. In some cases, a radiationdetector receives light from the sample as modulated by the aperture atdifferent locations, and acquires a plurality of M intensity imagescorresponding to the different aperture location. The M intensity imagescan be synthesized in the frequency domain to recover a complex,improved resolution image of the sample. In one aspect, opticalaberrations and misalignments in the optical system(s) may be estimatedand corrected through simulated annealing.

In certain aspects, an aperture-scanning Fourier ptychography imagingsystem comprises an aperture scanner that can generate an aperture at aplurality of N aperture locations at different times in an intermediateplane (e.g., Fourier plane) of an optical arrangement. In other aspects,an aperture-scanning Fourier ptychography imaging system comprises anaperture scanner that can generate a plurality of apertures that areshifted as a whole to a plurality of N locations at different times inan intermediate plane of the optical arrangement. Such a plurality ofapertures may be in pattern form (e.g., checkered pattern) or in anrandom order.

As used herein, an aperture can refer to an area in a plane that allowsincident light to pass to the next optical element in the opticalarrangement. In some cases, an area surrounding the aperture at thatplane may block/reflect or otherwise prevent incident light from passingto the next optical element. In certain aspects, the aperture may be anoptically transparent or substantially optically transparent area. Inthese aspects, the surrounding area may reflect or absorb the incidentlight. For example, the aperture may be a light transmissive region(e.g., hole) in an opaque plate. In other aspects, the aperture may areflective area (e.g., one or more micromirrors or one or morereflective pixels in a display) that reflects incident light to the nextoptical element. In these aspects, the surrounding area may eitherabsorb incident light or reflect incident light away from the nextoptical element. In one example, the aperture may be comprised of one ormore micromirrors oriented at an angle that reflects incident light tothe next optical element. In this example, one or more micromirrors inthe surrounding area may be oriented at a different angle that reflectslight away from the next optical element. In some cases, an aperturelocation may correspond to a centroid of the area of the aperture.

In certain aspects, aperture-scanning Fourier ptychography imagingsystems may comprise mechanically-based aperture scanners and/ordisplay-based aperture scanners. Certain mechancially-based aperturescanners can mechanically shift an aperture to different aperturelocations. In one case, a mechancially-based aperture scanner comprisesan X-Y translational stage that can translate/rotate a structure (e.g.,plate of opaque material having an aperture in the form of a lighttransmissive region such as a hole in the plate) having the aperture toshift the aperture to the plurality of aperture locations in theintermediate plane. Certain display-based aperture scanners candigitally generate an aperture at different locations, for example, bydisplaying an aperture and surrounding area on a display. Some examplesof display-based aperture scanners include a spatial light modulator(SLM) that generates an aperture and surrounding area on an SLM display.The SLM display may be, for example, a liquid crystal on silicon (LCoS)display or a digital micromirror device (DMD).

Certain aperture-scanning Fourier ptychographic systems and methodsdescribed herein may provide one or more technical advantages. Oneadvantage of certain systems is that they can be used for imaging ofthick and/or non-transmissive samples. Another advantage of certainssystems is that they can be adapted for luminescence (e.g.,fluorescence, phosphorescence, chemluminescence, bioluminescence, etc.)imaging.

Certain aperture-scanning Fourier ptychographic systems described hereincan be adapted for luminescence (e.g., fluorescence, phosphorescence,chemluminescence, bioluminescence, etc.) imaging. For example, certainsystems may be adapted to collect emissions directed back toward theillumination source.

In trans-illumination imaging configurations, a light detector mayacquire light data about light transmitted through the sample. Forexample, the illumination source may direct illumination toward thelight detector where the sample is located between the light detectorand the illumination source. In these trans-illumination imagingconfigurations, light reflected back toward the illumination source oremitted by the sample in the direction of the illumination source maynot be received by the light detector.

In fluorescence imaging and other luminescence imaging applications,fluorophores in the sample are excited by excitation illumination of acertain wavelength(s) from the illumination source and emit light of adifferent wavelength(s) (emissions). These emissions tend to have a weaksignal compared to the excitation light so that collection efficiencymay be important.

In some aspects, certain aperture-scanning Fourier ptychographic systemsmay be configured so that the light detector can receive emissions fromthe sample and/or light reflected from the sample back toward theillumination source. These systems have optical arrangements that canaccommodate an illumination source that directs excitation illuminationto the sample and away from next element in the system. In this way,propagation of the excitation illumination through the system may besubstantially avoided.

In some aspects, certain aperture-scanning Fourier ptychographic imagingsystems can be used to image thick and/or non-transparent samples. Inthese systems, a single arbitrarily patterned coherent illumination beammay be used to illuminate the sample from any direction. For thesesystems, there is a one-to-one relationship between each of theintensity images and different passbands of the 2D sample spectrum forboth thick and non-transparent samples. Thus, the recovery process canaccurately impose the panning spectrum constraint to recover ahigher-resolution (i.e. improved resolution) complex image of thickand/or non-transparent samples.

II. Optical Transfer Function Modulation in Ptychographic FourierImaging

In imaging systems, a sample may be illuminated by a light field and theoptical field E₁(x, y) emerging from the sample surface may be generallydescribed as: E₁(x, y)=A₁(x,y)e^(iφ1(x,y)). Certain ptychographicFourier imaging systems can be used to characterize E₁(x,y) anddetermine an aberration-free set of amplitude and phase data about thesample. In certain aspects, a ptychographic Fourier imaging system canbe used to determine a phase and amplitude distribution of the opticalfield E₁(x, y) to simultaneously correct for optical aberrations and/ormisalignments in the system as the sample is imaged.

An optical field E₁(x, y) may be transmitted through an optical systemto generate an optical field E₂(x, y)=O(E₁(x, y)) where O( ) representsthe optical transfer function performed on the light field by theoptical system. O( ) can be represented by any number of differentoperations. For example, O( ) can be represented as a Fourier transform(e.g., if the system is a simple lens with a sample at its focus planeand projection screen at infinity); it can be a unitary transformation(e.g., if the system is a perfect 4f system); or it can be a complexfunction. Optical aberrations are expressible within the opticaltransfer function. For example, a physical optical system may notperform a perfect Fourier transform, but its aberrations can bemathematically described as the ways it distorts the transform function.O( ) function may be fully characterizable by any number ofcharacterization means. Typically, the E₂(x,y) may be measured or putthrough additional optical systems prior to subsequent measurements.Suppose E₂(x,y) is measured by some means such as, for example, a lightdetector (e.g. digital camera). The intensity values measured may beexpressed as: |E₂(x,y)|². With only this amplitude measurement, it maynot be possible to apply an inverse function to get E₁(x,y). On theother hand, if both amplitude and phase knowledge of E₂(x,y) are knownand the function O( ) is known, then E₁(x,y) can be obtained by takingthe inverse O( ) function of E₂(x,y). That is, E₁(x,y)=O⁻¹(E₂(x,y)).

In certain aspects, Fourier ptychographic imaging systems withmodulation at the intermediate plane can be used determine bothamplitude and phase data of an optical field E₁(x, y) at the sampleplane. In some cases, modulation may be implemented by an aperturescanner.

FIG. 1 illustrates a schematic drawing of certain components of aFourier ptychographic imaging system 10 with optical transfer functionmodulation at an intermediate plane, according to embodiments. In oneexample, the optical function modulation may be implemented with anaperture scanner generating an aperture at N different locations at theintermediate plane such as, for example, a Fourier plane of the sampleplane of the optical system. In one case, the aperture scanning Fourierptychographic imaging system 10 may be able to determine amplitude andphase data of the optical field E₁(x,y) at a spatial resolution near orat the optical limit dictated by the numerical aperture (NA) of thesystem 10.

In FIG. 1, the aperture-scanning Fourier ptychographic imaging system 10comprises a first optical system 100 with an optical transfer functionof O_(A)( ) and a second optical system 200 with an optical transferfunction of O_(B)( ) According to the schematically represented lightfields in FIG. 1, the optical field E₁(x, y) from the sample is receivedby the first optical system 100. The resulting light field function isgiven by E_(1A)(x,y)=O_(A)(E₁(x,y)). In some cases, rough estimates ofthe optical transfer functions O_(A)( ) and O_(B)( ) of the first andsecond optical systems 100, 200 respectively may be used as an initialstarting point in a joint optimization procedure to determine a moreaccurate complex optical transfer function estimate, in conjunction withthe running of the Fourier ptychography recovery algorithm, such asdescribed in Xiaoze Ou, Guoan Zheng and Changhuei Yang, Embedded pupilfunction recovery for Fourier ptychographic microscopy,” Optics Express22 (5), pp. 4960-4972 (2014), which is hereby incorporated by referencefor this description.

If this system did not have modulation at the intermediate pane, thelight field from the first optical system 100 would propagate to thesecond optical system 200, which would result in a final light fieldfunction of E_(1AB)(x, y)=O_(B)(O_(A)(E₁(x, y))). The intensitydistribution, I(x, y)=|E_(1AB)(x, y)|² of the final light field can bemeasured spatially at the detector plane. In this case, the measuredintensity distribution I(x, y) may not provide enough information todetermine E₁(x, y) or E_(1A)(x, y) since the phase information is notmeasured by the light detector (only the amplitude).

In FIG. 1, the aperture function or other known modulating function withan optical transfer function of O_(C1)( ) is applied at the intermediateplane. In this case, the light field modulated at the intermediate planepropagates to second optical system 200, which results in a final lightfield function of E_(1A, OC1)(x,y)=O_(C1)(O_(A)(E₁(x,y))).

The aperture-scanning Fourier ptychographic imaging system 10 can use aFourier ptychographic method to determine both amplitude and phase dataof E₁(x, y). First, a guess of E₁(x, y) is made designated asE_(1guess)(x, y). Next, the aperture function or other known modulatingfunction is applied at the intermediate plane of E_(1A)(x, y). Thisaperture function may be an optical transfer function designated asO_(C1)( ) and the new E_(1A, OC1)(x,y)=O_(C1)(O_(A)(E₁(x,y))). The newE_(1AB), O_(C1)(x,y)=O_(B)(O_(C1)(O_(A)(E₁(x,y)))). The |E_(1AB),O_(C1)(x,y)|² is determined by acquiring the intensity distribution atthe detector plane. Next, E_(1A, OC1),guess(x,y)=O_(B)(O_(C1)(O_(A)(E_(1guess)(x,y)))) and |E_(1AB),O_(C1,guess)(x,y)|² are computationally determined and |E_(1AB),O_(C1)(x, y)|² is compared to |E_(1AB), O_(C1,guess)(x, y)|². If thecomparison shows a difference (i.e., they are not equal to each other),a new E_(1guess)(x, y) is generated by modifying the currentE_(1guess)(x, y) based on known restrictions on E_(1A), O_(C1)(x, y) and|E_(1AB), O_(C1)(x, y)|². One strategy for modifying the guess isprovided below. This process of modification of E_(1guess)(x, y) isiterated by applying the aperture or other known function at a differentlocation at the plane of E_(1A)(x,y) (e.g., at O_(C1)( ), O_(C2)( ),O_(C3)( ), . . . ) until we have reached convergence where |E_(1AB),O_(Cn)(x, y)|² is equal (or substantively equal based on error functionmeasures) to |E_(1AB), O_(Cn,guess)(x, y)|² for all O_(Cn)( ) functionswhere n=1, 2, 3 . . . .

III. Aperture-Scanning Ptychographic Fourier Imaging

Certain aspects described herein pertain to aperture-scanning Fourierptychographic imaging systems, devices and methods. The Fourierptychographic imaging systems comprise an aperture scanner. In certainaspects, the aperture scanner can generate an aperture at a plurality ofN aperture locations at different times in an intermediate plane of anoptical arrangement. In other aspects, the aperture scanner can generatea plurality of apertures that are shifted as a whole to a plurality of Nlocations at different times in an intermediate plane of the opticalarrangement. The intermediate plane may be, for example, a Fourier planeconjugate the sample plane. The Fourier ptychographic imaging systemsfurther comprise a light detector at a detector plane that is configuredto acquire a plurality of M intensity images of the sample.

An aperture scanner can refer to one or more devices configured togenerate the aperture (or plurality of apertures) at a plurality of Nlocations at an intermediate plane. In certain cases, each intensityimage of the plurality of M intensity images acquired by the lightdetector corresponds to a different aperture location of the pluralityof N aperture locations. The number of aperture locations N and/ornumber of intensity images M may be in the range of 1 to severalthousand. In one case, N and/or M may be a value in a range from 1 to1000. In another case, N and/or M may be a value in a range from 1 to2000. In another case, N and/or M may be a value in a range from 1 to3000. In some examples, N=M.

Although the apertures described herein with reference to certainillustrations are rectangular in shape having dimensions of width l andheight h, other shapes such as a circular shape with radius r,triangular, etc., may be used. In addition, the aperture at differentlocations of the plurality of N aperture locations is described inexamples as being of constant shape and size. It would be understoodhowever that the aperture can be of varying sizes and shapes atdifferent aperture locations. In one case, the area of the aperture hasa size of 0.5 mm×0.5 mm. In another case, the area of the aperture has asize of 5 mm×5 mm.

The plurality of N aperture locations may be described in the form of aone-dimensional array, a two-dimensional matrix, a hexagonal array, etc.In some cases, the plurality of aperture locations may be atwo-dimensional matrix in the form of a rectilinear grid (e.g., squaregrid), a curvilinear grid, etc. If the plurality of N aperture locationsis in a rectilinear grid arrangement having dimensions m×n, then theaperture locations may be designated as (X_(i), Y_(j)), i=1 to m, j=1 ton and the number of aperture locations, N=m×n. If such a rectilineargrid has square dimensions of n×n, then the aperture locations may bedesignated as (X₁, Y_(j)), i=1 to n, j=1 to n and N=n².

The N aperture locations can be generated in any order (e.g.,sequential, random, row by row, column by column, etc.) over time duringthe image acquisition process. For example, a sequential column bycolumn order through a rectilinear grid may be: (X₁,Y₁), (X₁,Y₂),(X₁,Y₃), . . . (X₁,Y_(n)), (X₂,Y₁), (X₁,Y₂), (X₁,Y₃), . . . (X₂,Y_(n)),. . . (X_(m),Y_(n)) at sample times t_(i)=1 to M, where M=m×n.Alternatively, a random order may be used.

In certain aspects, the plurality of N aperture locations includes anoverlapping area between two or more of its neighboring apertures (i.e.apertures at adjacent aperture locations). In one example, theoverlapping area may be about 70% of the aperture area. In anotherexample, the overlapping area may be about 75% of the aperture area. Inanother example, the overlapping area may be between about 2 and 90% ofthe aperture area. In some cases, particular values of m and n may beused so that neighboring apertures overlap by a predefined amount (e.g.,70%, 75%, etc.).

In some aspects, mechancially-based aperture scanners can mechanicallyshift an aperture to different aperture locations. For example, amechanically-based aperture scanner may comprise an X-Y stage configuredto physically translate and/or rotate a structure having an aperture(e.g., plate of opaque material having an aperture in the form of alight transmissive region such as a hole in the plate) to generate theaperture at the different aperture locations. In one example, a platewith an aperture may be affixed to the X-Y stage and the X-Y stage maythen translate and/or rotate the plate in the intermediate plane tolocate the aperture at the appropriate aperture locations at thecorresponding acquisition times. In one case, the plate may have asurface with the aperture located orthogonal to the surface. The X-Ystage may translate/rotate the plate so that the surface remains in theintermediate plane.

In some aspects, display-based aperture scanners can digitally displayan aperture at different aperture locations. An example of adisplay-based aperture scanner is a spatial light modulator or SLM. A“spatial light modulator” or “SLM” can refer to a device(s) that cangenerate an aperture on its display. In some cases, an SLM uses anelectrical and/or optical signal from an SLM light source to modulatephase, φ, and/or amplitude of light. In some cases, the SLM light sourcemay be a collimated light source such as a laser (e.g., Excelsior® 532SM). In other cases, the SLM light source may not be collimated light.For example, the light may be spatially filtered light from a lightemitting diode (spatial coherence length of approximately 1 mm, spectralbandwidth of 20 nm), or light from a laser source (e.g., 532 nmquasi-monochromatic laser light, spatial coherence length of multiplemeters). The SLM light source may be a component of theaperture-scanning Fourier ptychographic imaging system or may be aseparate component. Certain SLMs may be commercially available. Incertain aspects, an SLM comprises an SLM display having a plurality ofSLM display elements. Each SLM display element can be set to function asan aperture (aperture setting) or to function as the area surroundingthe aperture (field setting). In some configurations, an SLM displayelement in an aperture setting is transparent or nearly transparent topass incident light and a display element in a field setting mayblock/reflect or nearly bock/reflect incident light. In otherconfigurations, certain SLM display elements may be reflective. In thesecases, a display element in the aperture setting is oriented at a(first) angle to reflect incident light to the next optical element inthe optical arrangement and a display element in a field setting isoriented at a different (second) angle that reflects incident light awayfrom the next optical element. In these configurations, the SLM displaycan generate an aperture at one or more SLM display elements by settingthese display elements in an aperture setting and/or setting thesurrounding display elements in a field setting. At differentacquisition times, t_(i), different sets of one or more display elementsare at appropriate settings to generate the aperture at thecorresponding aperture location. In some cases, the SLM display may havea refresh rate in the range of 30 per second to 100 per second.

In aperture-scanning Fourier ptychographic imaging systems comprising anaperture scanner in the form of an SLM, different types of SLM displaysmay be used such as, for example, a reflective liquid-crystal on silicon(LCoS) display, a digital micromirror device (DMD), etc. A reflectiveliquid-crystal on silicon (LCoS) display is a reflective display havinga plurality of reflective display elements. An example of a commerciallyavailable LCoS display is the reflective HOLOEYE® SLM, Pluto, phase onlyLCoS, 8 μm pixel size, 1080×1920 pixels display. A DMD can refer to anoptical semiconductor chip having on its surface multiple microscopicmicromirrors. In certain aspects, each micromirror can be individuallyrotated to an angle, a. In this way, each micromirror can betransitioned to either an aperture setting at angle, a, or to a fieldsetting at no rotation, or visa versa Although these micromirrors areusually arranged in a rectangular array (dimensions o×p), otherarrangements may be used. In certain aspects, each micromirror of theDMD may correspond to one or more light detector pixels. In one case,one or more of the micromirrors in the aperture setting may be orientedso that an optical axis orthogonal to the surface of the micromirror isoriented at an angle, a, from the Fourier plane. An example of this caseis shown in FIGS. 4 and 5.

In aperture-scanning Fourier ptychographic imaging systems comprising anaperture scanner in the form of an SLM, the SLM display may be locatedso that its display plane at the intermediate plane (e.g., Fourierplane). In some cases, the SLM display may be in the form of atwo-dimensional matrix of display elements (e.g. pixels) at the displayplane. The two-dimensional matrix has dimensions of Pix₁×Pix₂, wherePix₁ is the number of pixels in a first direction and Pix₂ is the numberof pixels in a second direction orthogonal to the first direction. Inone example, the SLM display is a 1920-by-1080 pixel display where Pix₁is 1920 and Pix₂ is 1080. In certain aspects, the display elements ofthe SLM are programmed to have particular settings at differentacquisition times according to illumination instructions.

A sample being imaged by aperture-scanning Fourier ptychographic imagingsystems may be comprised of one or more objects or one or more portionsof an object. Each object may be a biological entity or an inorganicentity. Examples of biological entities include whole cells, cellcomponents, microorganisms such as bacteria or viruses, cell componentssuch as proteins, thin tissue sections, etc. In some cases, the samplemay be provided in a medium such as a liquid.

In luminescence imaging examples, a reagent (e.g.,fluorescence/phosphorescence dye) may be mixed with the sample to markor tag portions under investigation with fluorophore. A fluorophore canrefer to a component of a molecule that causes the molecule to fluoresceor phosphoresce. A fluorophore can absorb energy from excitation lightof a specific wavelength(s) and re-emit the energy at a differentwavelength(s). In luminescence imaging examples, the illumination sourceilluminates the sample with excitation light of predeterminedwavelength(s) (e.g., blue light) to activate the fluorophore in thesample. In response, the fluorophore release emissions of a differentwavelength(s) (e.g., red light).

In certain aspects, an illumination source(s) provides illumination tothe sample being imaged by an aperture-scanning Fourier ptychographicimaging system. The illumination source may be a component of orseparate from the aperture-scanning Fourier ptychographic imagingsystem. Although the illumination source is described in some cases asbeing located to direct illumination toward the first optical element inthe optical arrangement, the illumination source may be located in otherlocations to direct illumination away from the first optical element.For example, in a luminescence imaging example, the illuminationsource(s) may provide excitation light that is directed away from thefirst optical system in the optical arrangement. In many cases,excitation illumination has a stronger signal than emissions from thesample. By directing the excitation illumination away from the firstoptical system, this configuration will aid in collecting a weakeremissions signal by the light detector. Although a single illuminationsource is described in many cases, it would be understood that multipleillumination sources may be used.

In certain cases, the aperture-scanning Fourier ptychographic imagingtechniques pertain to a sample illuminated by a single arbitrarilypatterned coherent illumination beam from any direction. In many cases,the angle of illumination does not vary during the image acquisitionprocess. In some cases, the illumination may be monochromatic. Inanother case, the illumination source may provide illumination ofdifferent wavelengths (e.g., wavelengths associated with RGB) atdifferent acquisition times as discussed below. Although theillumination source(s) may be coherent source(s), incoherent source(s)may also be used and computational corrections may be applied. Someexamples of a source of visible light include an LCD pixel and a pixelof an LED display. In cases that use other forms of radiation, othersources of radiation may be used. For example, in embodiments that useX-ray radiation, the radiation source may comprise an X-ray tube and ametal target. As another example, in cases that use microwave radiation,the radiation source may comprise a vacuum tube. As another example, incases that use acoustic radiation, the radiation source may be anacoustic actuator. As another example, in cases that use Terahertzradiation, the radiation source may be a Gunn diode. One skilled in theart would contemplate other sources of radiation.

In color imaging implementations, the illumination source may provideRGB illumination of three wavelengths λ1, λ2, and λ3 corresponding tored, green, blue colors, respectively. In one case that uses Terahertzradiation, the frequencies of the radiation provided by the illuminationsource may be in the range of 0.3 to 3 THz. In one case that usesmicrowave radiation, the frequencies of the radiation provided by thevariable illuminator may be in the range of 100 MHz to 300 GHz. In onecase that uses X-ray radiation, the wavelengths of the radiationprovided by the variable illuminator may be in the range of 0.01 nm to10 nm. In one case that uses acoustic radiation, the frequencies of theradiation provided by the variable illuminator may be in the range of 10Hz to 100 MHz.

In certain aspects, a “radiation detector” or “light detector” or“detector” is configured to acquire an intensity image of the sample bymeasuring/recording an intensity distribution of incident radiation at adetector plane at a particular sample (acquisition) time. During animage acquisition process, for example, the radiation detector mayacquire a plurality of M intensity images at M sample times,t_(i-1 to M). If visible light radiation is being measured, theradiation detector may be in the form of a charge coupled device (CCD),a CMOS imaging sensor, an avalanche photo-diode (APD) array, aphoto-diode (PD) array, a photomultiplier tube (PMT) array, or likedevice. If using THz radiation is detected, the radiation detector maybe, for example, an imaging bolometer. If using microwave radiation isused, the radiation detector may be, for example, an antenna. If usX-ray radiation is used, the radiation detector may be, for example, anx-ray sensitive CCD. If using acoustic radiation is used, the radiationdetector may be, for example, a piezoelectric transducer array. Theseexamples of radiation detectors and others are commercially available.In some cases, the radiation detector may be a color detector e.g. anRGB detector. In other cases, the radiation detector need not be a colordetector. In certain cases, the radiation detector may be amonochromatic detector.

A “sample” or “acquisition” time can refer to a time that the lightdetector captures an intensity image of the sample. During certain imageacquisition processes described here, the radiation detector captures aplurality of M intensity images (e.g., M=1, 2, 5, 10, 20, 30, 50, 100,1000, 10000, etc.). At each sample time, t_(i), that an intensity imageis captured, the aperture is at a different scanning location of theplurality of N aperture locations. In certain cases, the sampling ratesmay range from 0.1 to 1000 frames per second.

In certain aspects, the radiation detector may have discrete radiationdetecting elements (e.g., pixels). The radiation detecting elements maybe of any suitable size (e.g., 1-10 microns) and any suitable shape(e.g., circular, rectangular, square, etc.). For example, a CMOS or CCDelement may be 1-10 microns and an APD or PMT light detecting elementmay be as large as 1-4 mm. In one example, the radiation detectingelement is a square pixel having a size of 5.5 um.

The radiation detector generates image data comprising the plurality ofM intensity images. The radiation detector may also generate other imagedata such as the sample times and other related data.

Fourier space can refer to a mathematical space spanned by wavevectorsk_(x) and k_(y), being the coordinate space in which the two-dimensionalFourier transforms of the spatial images created by theaperture-scanning Fourier ptychographic imaging system reside. Fourierspace may also refer to the mathematical space spanned by wavevectorsk_(x) and k_(y) in which the two-dimensional Fourier transforms of thespatial images collected by the radiation sensor reside.

Each of the plurality of M intensity images captured by the radiationdetector is associated with a region in Fourier space. In Fourier space,neighboring regions may share an overlapping area over which they samplethe same Fourier domain data. This overlapping area in Fourier spacecorresponds to the overlapping area of neighboring apertures in theintermediate plane. In certain aspects, the plurality of N aperturelocations is designed so that the overlapping area of neighboringaperture locations will generate a certain amount of overlapping area inthe Fourier domain data. In one case, the plurality of aperturelocations are designed to generate an overlapping area in the Fourierdomain data in the range of about 2% to about 99.5% of the area of oneof the regions. In another embodiment, the overlapping area betweenneighboring regions may have an area that is in the range of 65% to 75%the area of one of the regions. In another embodiment, the overlappingarea between neighboring regions may have an area that is about 65% ofthe area of one of the regions.

FIGS. 2A and 2B are schematic drawings of components of anaperture-scanning Fourier ptychographic imaging system 11, according toembodiments. In this illustration, the optical elements are in a 4foptical arrangement and aperture scanning is at a Fourier plane of thesample. The aperture-scanning Fourier ptychographic imaging system 11comprises a first optical system (e.g., lens) 101 having a first focallength f₁ (where f₁=f) a second optical system (e.g., lens) 201 having asecond focal length f₂ (where f₁=f), and an aperture scanner 300. Theaperture-scanning Fourier ptychographic imaging system 11 also includesa sample plane, a detector plane, and a Fourier plane of the sample (notshown). During image acquisition, a sample being imaged is located atthe sample plane. Although not shown, the aperture-scanning Fourierptychographic imaging system 11 further comprises a detector at thedetector plane. Optionally, the aperture-scanning Fourier ptychographicimaging system 11 may further comprise an illumination source forilluminating the sample. Also optionally, the aperture-scanning Fourierptychographic imaging system 11 may further comprise one or morecomponents of a computing device comprising a processor, a display incommunication with the processor, and a computer readable medium.

According to the 4f optical arrangement shown in FIGS. 2A and 2B, thefirst optical system 101 is located at a distance from the secondoptical system 201 equal to their combined focal lengths 2f. The sampleplane is located at an optical path distance of the first focal lengthfrom the first optical system 101 and the detector plane is located anoptical path distance of a second focal length from the second opticalsystem 201. The Fourier plane of the sample is located at distance of afirst focal length from the first optical system 101 and located at anoptical path distance of the second focal length from the second opticalsystem 201.

The illustrated aperture-scanning Fourier ptychographic imaging system11 also includes an aperture 310. The aperture-scanning Fourierptychographic imaging system 11 may further comprise an aperture scanner300 configured to provide the aperture 310 at a plurality of aperturelocations in the Fourier plane.

In this illustrated example, the aperture is shown at two neighboringaperture locations at different sampling times. FIG. 2A shows theaperture 310(a) at a first aperture location. FIG. 2B shows aperture310(b) at a second aperture location. FIG. 2B also shows aperture 310(a)in a dotted line to illustrate the overlapping region 312 between thetwo adjacent aperture locations.

During certain image acquisition processes, the aperture scanner of anaperture-scanning Fourier ptychographic imaging system generates anaperture at a plurality of N aperture locations (X_(i), Y_(j)), i=1 ton, j=1 to m, M=n×m. At neighboring aperture locations in the pluralityof aperture locations there is an overlapping region (e.g., 312) betweenneighboring aperture locations. At the detector plane, the lightdetector acquires an intensity image while the aperture is at acorresponding aperture scanning position. During the image acquisitionprocess, the light detector acquires a plurality of M intensity imagescorresponding to different aperture locations. The M intensity images(i.e. I_(i,j), i=1 to o, j=1 to p and M=o×p) are acquired at thedetector plane at acquisition times, t_(i,j), i=1 to o, j=1 to p. Thenumber of intensity images, M, acquired by the light detector can be inthe range of 1 to a few thousand intensity images. During certain imagerecovery processes, an aperture-scanning Fourier ptychographic imagingsystem recovers a higher resolution, complex field E₁(x, y) at thesample plane from the plurality of M intensity images. In certainaspects, the complex field at the sample plane can then be propagated tovarious planes (e.g., planes parallel to the sample plane). Thesepropagated images can be used to form a 3D image of an extended sample.

Details of certain Fourier ptychographic acquisition and recoveryprocesses can be found in Section IV below. An example of an Fourierptychographic recovery process can also be found in Guoan Zheng, RoarkeHorstmeyer, and Changhuei Yang, “Wide-field, high-resolution Fourierptychographic microscopy,” Nature Photonics 6, pp. 739-745 (2013), whichis hereby incorporated by reference in its entirety. Certain details ofan aperture-scanning Fourier ptychographic imaging system can be foundin Dong, Siyuan et al., “Aperture-scanning Fourier ptychography for 3Drefocusing and super-resolution macroscopic imaging,” pp. 13586-13599(Jun. 2, 2014), which is hereby incorporated by reference in itsentirety.

There may be similarities between certain components of theaperture-scanning Fourier ptychographic imaging system 11 in FIGS. 2Aand 2B, the aperture-scanning Fourier ptychographic imaging system 12 inFIG. 3A, the aperture-scanning Fourier ptychographic imaging system 14in FIG. 4, the aperture-scanning Fourier ptychographic imaging system 15in FIG. 5, the aperture-scanning Fourier ptychographic imaging system 16in FIG. 6, and the aperture-scanning Fourier ptychographic imagingsystem 17 in FIG. 7.

FIG. 3A is a schematic drawing of components of an aperture-scanningFourier ptychographic imaging system 12, according to embodiments. Theaperture-scanning Fourier ptychographic imaging system 12 comprises afirst optical system (e.g., lens) 102 having a first focal length f₁(where f₁=f) a second optical system (e.g., lens) 202 having a secondfocal length f₂ (where f₁=f), and an aperture scanner 302 in the form ofa spatial light modulator. The aperture scanner 302 is configured toshift an aperture 310 to a plurality of N locations at an intermediateplane such as the Fourier plane of the sample 50. Although the aperturescanner 302 is illustrated in the form of a spatial light modulator, itwould be understood that other types of aperture scanners could be used.The illustration shows the system during the image acquisition processwith a sample 50 being imaged located at a sample plane. Theaperture-scanning Fourier ptychographic imaging system 12 furthercomprises a detector 500 with a (active) detecting surface at a detectorplane.

Some details of an aperture-scanning Fourier ptychographic imagingsystem using a spatial light modulator for shifting an aperture can befound in Horstmeyer, Roarke et al., “Overlapped Fourier coding foroptical aberration removal,” (2014), which is hereby incorporated byreference in its entirety.

The aperture-scanning Fourier ptychographic imaging system 12 furthercomprises an optional illumination source 400 that can provideillumination 410 to the sample 50. The illumination source 400 mayprovide a single arbitrarily patterned coherent illumination beam fromany direction. Although illumination source 400 is shown in a locationproviding illumination 410 toward the light detector 500 intrans-illumination configuration, the illumination source 400 may be inother locations to provide illumination 410 in other directions or othercomponents (e.g., reflective elements) may be used to directillumination in other directions, such as, away from the next opticalelement, for example, the first optical system 102. Also optionally, theaperture-scanning Fourier ptychographic imaging system 11 may furthercomprise one or more components of a computing device comprising aprocessor, a display in communication with the processor, and a computerreadable medium.

In FIG. 3A, the aperture-scanning Fourier ptychographic imaging system12 is in a 4f optical arrangement with the first optical system 102located at a distance from the second optical system 202 equal to theircombined focal lengths 2f. The sample plane of the sample 50 is locatedat the first focal length (f₁=f) from the first optical system 102 andthe detector plane of the detector 500 is located at an optical pathdistance of the second focal length (where f₂=f) from the second opticalsystem 202. The Fourier plane of the sample is located at an opticalpath distance of the first focal length (where f₁=f) of the firstoptical system 102 away from the first optical system 102 and located atan optical path distance of the second focal length (where f₂=f) of thesecond optical system 202 away from the second optical system 202.

FIG. 3B is a schematic drawing of cross-sectional view of an SLM display323 of a spatial light modulator 303 that can be implemented in certainaperture-scanning Fourier ptychographic imaging systems describedherein. The cross-sectional view is at a display plane of the SLMdisplay 323. FIG. 3B includes an x′-axis and a y′-axis at the displayplane. The spatial light modulator 303 described with respect to FIG. 3Bmay be similar in some respects to the aperture scanner 302 describedwith respect to FIG. 3A.

In FIG. 3B, the SLM display 323 is a rectangular display with dimensionsof width L and height H. The spatial light modulator 303 may beconfigured (e.g. programmed) to digitally generate on its display 323the aperture 310 at a plurality of N locations. In this example, theplurality of N aperture locations is in the form of a 2-D rectilineargrid with equally-spaced locations (i.e. equal spacing betweenneighboring apertures). In other embodiments, the spacing betweenneighboring aperture locations may not be equally spaced and/or theaperture may have different sizes at different locations.

In FIG. 3B, the display 303 is shown at acquisition time, t₁, when anaperture 310(1) (shown in sold line) is generated on the SLM display323. The illustration also includes a neighboring aperture 310(2) (shownin dotted line) that is displayed at another acquisition time (e.g., t₂)as denoted by a dotted line to illustrate the spatial overlappingrelationship between the neighboring apertures. As shown, neighboringapertures 310(1), 310(2) have an overlap 312 in the x′-direction of adistance c.

In some cases, the overlap 312 may be at least about 70% of the area ofthe aperture 310. In other cases, the overlap 312 may be at least about75% of the area of the aperture 310. In other cases, the overlap 312 maybe between 2-90% of the area of the aperture 310. Display instructionsmay be used by the SLM 303 to generate an aperture on the display 323 inthe rectilinear grid.

The overlap 312 between neighboring (adjacent) apertures may correspondto setting the n>L/l. For example, if n=9, setting L/l=2.5 will generatean overlap between neighboring apertures of more than 75%. Bothapertures 310(1) and 310(2) have a constant rectangular shape with awidth l and height of h. In other embodiments, the aperture 310displayed at different locations may have different sizes and/or shapes.

In FIG. 3B, the SLM display 303 has a 2-D rectilinear grid with squaredimensions (n×n dimensions). In this case, the N aperture locations aredescribed as (X_(i), Y_(j)), i=1 to n, j=1 to n, in the display planeand the number of aperture locations, N=n². Typically, the aperture 310may be displaced from the origin of this 2-D rectilinear grid by atwo-dimensional vector c_(j)=(c_(xj), c_(yj)) for 1<j<n². In thisconfiguration, a light detector can capture at the detector plane Mdifferent intensity images, I_(k,l), (M=k×l) at different aperturelocations and corresponding acquisition times.

FIGS. 4-6 are schematic drawings illustrating examples of differentconfigurations of the components of the aperture-scanning Fourierptychographic imaging system 12 described with reference to FIG. 3A.

FIG. 4 is a schematic drawing of components of an aperture-scanningFourier ptychographic imaging system 14, according to certain aspects.The aperture-scanning Fourier ptychographic imaging system 14 comprisesa first optical system (e.g., lens) 102 having a first focal lengthf₁=f, a second optical system (e.g., lens) 202 having a second focallength f₂=f, and a detector 500. The aperture-scanning Fourierptychographic imaging system 14 further comprises an aperture scannercomprising a DMD array 320 having a display surface 322 and a sequenceof one or more mirrors 330 having a reflective surface 332. The surface322 includes a y′-axis and an x′-axis (not shown) orthogonal to they′-axis, both in the plane at the surface 322. The illustrated exampleis shown with a sample 50 being imaged at a sample plane.

The aperture-scanning Fourier ptychographic imaging system 14 alsocomprises an optional illumination source 400 configured to provideillumination 410 to the sample 50 during an image acquisition process asshown in the illustration. In this illustrated example, the illuminationsource 400 is shown located (e.g., between first optical system 102 andthe sample 50) to direct illumination 410 away from the first opticalsystem 102. In the configuration, the first optical system 102 canreceive light reflected from the sample surface or emitted from thesample 50. The illustrated configuration can be used in luminescenceimaging applications. In other examples, the illumination source 400 maybe in other locations and/or direct illumination in other directions.Although a single illumination source 400 is shown in this example,multiple illumination sources may be used.

The aperture-scanning Fourier ptychographic imaging system 14 is in a 4foptical arrangement with the first optical system 102 located at anoptical path distance from the second optical system 202 equal to theircombined first and second focal lengths 2f. The sample plane is locatedat an optical path distance of the first focal length f₁=f from thefirst optical system 102.

In this 4f arrangement, the detector plane is located at an optical pathlength of the second focal length f₂=f from the second optical system202. The DMD array 320 is located at an optical path length of the firstfocal length f₁=f away from the first optical system 102. The sequenceof one or more mirrors 330 is located at an optical path length, b, fromthe DMD array 320 and at an optical path length, a, from the secondoptical system 202. The combined optical path distance between the DMDarray 320 and the second optical system 202 is a +b=f. The Fourier planeof the sample is located at an optical path length of the first focallength f₁=f of the first optical system 102 away from the first opticalsystem 102 and located at a combined optical path length a +b=f from thesecond optical system 202. In FIG. 4, a sample 50 being imaged is shownlocated at the sample plane, the detector is located so that the activedetecting surface is at the detector plane, and aperture scanner 320 islocated so that the display surface 322 is at the Fourier planeassociated with the sample plane of the sample 50.

The DMD array 320 is configured to shift an aperture to a plurality of Naperture locations at the Fourier plane of the sample 50. The DMD array320 comprises a plurality of micromirrorrs. The DMD array 320 generatesan aperture at each aperture location at the display surface by rotatinga corresponding set of one or more micromirrors of the DMD array 320 toreflect incident light at an angle, a, directed to the one or moremirrors 330. In some cases, other surrounding micromirrors in the DMDarray 320 are oriented at an angle that reflects incident light awayfrom the one or more mirrors 330.

In FIG. 4, the one or more mirrors 330 are configured to receive lightreflected by the aperture generated by the DMD array 320 to secondoptical system 202. In some aspects, the sequence of one or more mirrors330 may be configured to correct the differences in optical path lengthat the different locations along the y′-axis to the surface of themirrors 330. The illustration indicates an optical path b of a centerray between the surface 322 of the DMD array 320 and the surface 332 ofthe mirror(s) 330 and the optical path length a between the mirror(s)330 and the second optical system 202. The combined optical path of thecenter ray between first optical system 102 and the second opticalsystem is a+b=f. However, the optical path distance between the sequenceof mirrors 330 and the DMD array 320 is not the same from edge to edgeof these devices. To correct these differences, the sequence of one ormore mirrors 330 may have locations and/or orientations that correct forthese differences. For example, a binary grating pattern (i.e., a blazedgrating) may be super-imposed on top of the sub-aperture patterndisplayed on the DMD. Alternatively, an algorithm similar to thesimulated annealing correction approach discussed in Horstmeyer, Roarkeet al., “Overlapped Fourier coding for optical aberration removal,”(2014) may be used to find an arbitrarily-shaped pattern of mirrors tooffer optimized correction performance. This reference is herebyincorporated by reference in its entirety for details of this approach.

Although not shown, the aperture-scanning Fourier ptychographic imagingsystem 14 may also include one or more components of a computing device,which comprises a processor, a display in electrical communication withthe processor, and a computer readable medium in electricalcommunication with the processor.

FIG. 5 is a schematic drawing of components of an aperture-scanningFourier ptychographic imaging system 15, according to certain aspects.The aperture-scanning Fourier ptychographic imaging system 15 comprisesa first optical system (e.g., lens) 102 having a first focal lengthf₁=f, a second optical system (e.g., lens) 202 having a second focallength f₂=f, and a detector 500. The aperture-scanning Fourierptychographic imaging system 14 further comprises an aperture scanner inthe form of a DMD array 320 having a display surface 322. The surface322 includes a y′-axis and an x′-axis (not shown) orthogonal to they′-axis, both in the plane at the surface 322. The illustrated exampleis shown with a sample 50 being imaged at a sample plane.

The aperture-scanning Fourier ptychographic imaging system 15 alsocomprises an optional illumination source 400 configured to provideillumination 410 to the sample 50 during an image acquisition process asshown in the illustration. For example, illumination source 400 mayprovide a single arbitrarily patterned coherent illumination beam fromany direction. In this illustrated example, the illumination source 400is shown located (e.g., between first optical system 102 and the sample50) to direct illumination 410 away from the first optical system 102.In the configuration, the first optical system 102 can receive lightreflected from the sample surface or emitted from the sample 50. Theillustrated configuration can be used in luminescence imagingapplications. In other examples, the illumination source 400 may be inother locations and/or direct illumination in other directions. Althougha single illumination source 400 is shown in this example, multipleillumination sources may be used.

In this configuration, the angle, θ, between the center ray opticalpaths between first optical system 102 and the DMD array 320 and thesecond optical system 202 and the DMD array 320 is small angle. Sincethe angle, θ, is small in this configuration, the optical path distancesfor these center rays can be approximated as parallel and of equaldistances. In one case, the angle, θ, may be between about 1 degree andabout 10 degrees. In another case, the angle, θ, is about 10 degrees. Inanother case, the angle, θ, is about 15 degrees.

With this above-discussed approximation, the aperture-scanning Fourierptychographic imaging system 14 is approximated as a 4f opticalarrangement with the first optical system 102 located at an optical pathdistance from the second optical system 202 that is approximated asequal to the combined first and second focal lengths 2f. The sampleplane is located at the first focal length f₁=f from the first opticalsystem 102 and the detector plane is located at the second focal lengthf_(s)=f from the second optical system 202. The Fourier plane of thesample is located at an optical path length of the first focal lengthf₁=f of the first optical system 102 away from the first optical system102 and located at an optical path length of approximately the secondfocal length f₂=f of the second optical system 202 away from the secondoptical system 202.

In FIG. 5, a sample 50 being imaged is shown located at the sample planeand the detector 500 is located so that the active detecting surface isapproximately at the detector plane. The DMD array 320 is located at anoptical path length of the first focal length f₁=f away from the firstoptical system 102 and located at an optical path length ofapproximately the second focal second focal length f₂=f from the secondoptical system 202.

The DMD array 320 is configured to shift an aperture to a plurality of Naperture locations at the Fourier plane of the sample 50. The DMD array320 comprises a plurality of micromirrorrs. The DMD array 320 generatesan aperture at each aperture location at the display surface by rotatinga corresponding set of one or more micromirrors of the DMD array 320 toreflect incident light at an angle, a, directed to the second opticalsystem 202. In some cases, other surrounding micromirrors in the DMDarray 320 are oriented at an angle that reflects incident light awayfrom the second optical system 202.

Although not shown, the aperture-scanning Fourier ptychographic imagingsystem 14 may also include one or more components of a computing device,which comprises a processor, a display in electrical communication withthe processor, and a computer readable medium in electricalcommunication with the processor.

FIG. 6 is a schematic drawing of a view of components of anaperture-scanning Fourier ptychographic imaging system 16, according tocertain aspects. The aperture-scanning Fourier ptychographic imagingsystem 16 comprises a first optical system (e.g., lens) 102 having firstoptical system (e.g., lens) 102 having a first focal length f₁=f, asecond optical system (e.g., lens) 202 having a second focal lengthf₂=f, and a detector 500. The aperture-scanning Fourier ptychographicimaging system 16 further comprises an aperture scanner. In thisillustrated example, the aperture scanner comprises a beam splitter 340,a LCOS array 350 having a display surface 352, and a mirror 360. Thesurface 352 includes a y′-axis and an x′-axis (not shown) orthogonal tothe y′-axis.

The aperture-scanning Fourier ptychographic imaging system 16 alsocomprises an optional illumination source 400 configured to provideillumination 410 to the sample 50 during an image acquisition process asshown in the illustration. In this illustrated example, the illuminationsource 400 is shown located (e.g., between first optical system 102 andthe sample 50) to direct illumination 410 away from the first opticalsystem 102. In the configuration, the first optical system 102 canreceive light reflected from the sample surface or emitted from thesample 50. The illustrated configuration can be used in luminescenceimaging applications. In other examples, the illumination source 400 maybe in other locations and/or direct illumination in other directions.Although a single illumination source 400 is shown in this example,multiple illumination sources may be used.

The aperture-scanning Fourier ptychographic imaging system 16 is in a 4foptical arrangement with the first optical system 102 located at anoptical path distance from the second optical system 202 equal to theircombined first and second focal lengths 2f. The sample plane is locatedat an optical path distance of the first focal length f₁f from the firstoptical system 102. In this 4f arrangement, the detector plane islocated at an optical path length of the second focal length f₂=f fromthe second optical system 202. The LCOS array 350 is located at anoptical path length of the first focal length f₁f away from the firstoptical system 102.

The beam splitter 340 is configured to pass incident light of firstwavelength(s) received from the first optical system 102 and toabsorb/reflect incident light of second wavelength(s) received from thefirst optical system 102. For example, the beam splitter 340 may beconfigured to pass incident light of wavelengths associated withemissions from fluorophore in a sample illuminated by excitationillumination in a fluorescent imaging application. The beam splitter 340is further configured to absorb incident light of the secondwavelength(s) received from the LCOS array 350, and reflect incidentlight of the first wavelength(s) received from the LCOS array 350 to themirror 360. Alternatively, a conventional beam splitter may be used withthe addition of a spectral filter placed anywhere in the optical pathbetween the sample and the detector, which can pass light of wavelengthsassociated with emissions from fluorophore and absorb excitationillumination in a fluorescent imaging application.

In FIG. 6, the optical path distance between the LCOS array 350 and thebeam splitter 340 is designated as, a. The optical path distance betweenthe beam splitter 340 and the mirror 360 is b. The optical path distancebetween the mirror 360 and the second optical system 202 is c. Thecombined optical path distance between the LCOS array 350 and the secondoptical system 202 is a +b+c=f. The Fourier plane of the sample in thisoptical arrangement is at an optical path length of the first focallength f₁=f from the first optical system 102 and located at a combinedoptical path length a +b+c=f from the second optical system 202. In FIG.4, a sample 50 being imaged is shown located at the sample plane, thedetector 500 is located so that the active detecting surface is at thedetector plane, and display surface 352 of the LCOS array 350 is locatedat the Fourier plane associated with the sample plane.

Advantages of this configuration may be that the optical path is ofequal length between the first and second optical systems 102 and 202and that the optical elements do not need to be placed at challengingangles.

The LCOS array 350 is configured to shift an aperture to a plurality ofN aperture locations at an intermediate plane, which in this case is theFourier plane associated with the sample plane. The LCOS array 350comprises display comprised of a plurality of display elements that canbe set to be reflective. The LCOS array 350 generates an aperture ateach aperture location at the display surface by setting one or moredisplay elements to be reflective in order to reflect incident lightback to the beam splitter 340. In some cases, the surrounding elementsare set to be substantially transmissive or absorbtive.

Although certain aperture-scanning Fourier ptychographic imaging systemsare described as configured to generate an aperture at an intermediateplane, a plurality of apertures may be generated instead.

Although not shown, the aperture-scanning Fourier ptychographic imagingsystem 16 may also include one or more components of a computing device,which comprises a processor, a display in electrical communication withthe processor, and a computer readable medium in electricalcommunication with the processor.

IV. Aperture-Scanning Fourier Ptychographic Imaging System

In certain aspects, an aperture scanning Fourier ptychographic systemcomprises a first optical system (e.g., first lens), an aperture scannerconfigured to generate an aperture at a plurality of N aperturelocations at an intermediate plane, a second optical system (e.g.,second lens), and a radiation detector configured to capture a pluralityof M intensity images. Optionally, the aperture scanning Fourierptychographic system may further comprise an illumination source forproviding illumination and/or a processor. In some cases, theillumination source may provide a single arbitrarily patterned coherentillumination beam from any direction. In certain aspects, the firstoptical system, second optical system, radiation detector, and sampleplane of the sample are arranged in a 4-f optical arrangement. During animage acquisition process, the illumination source illuminates a sampleplaced at a sample plane. The first optical system receives light fromthe sample and the aperture scanner generates an aperture at a pluralityof locations at the Fourier plane of the sample. There is an overlappingarea between apertures at certain adjacent locations of the plurality ofN aperture locations. The second optical system receives light throughthe aperture. The radiation detector receives light from the secondoptical system as modulated by the aperture at the different locations.The radiation detector captures a plurality of M intensity imagescorresponding to different aperture locations of the plurality of Naperture locations. During a recovery process, a processor iterativelystitches together the M overlapping intensity images in Fourier space torecover a wide-field, complex image of the sample. In certain aspects,the aperture scanning Fourier ptychographic system can also digitallyadjust the complex higher-resolution image to accommodate for defocusand correct aberrations in its optical elements. In certain cases, theaperture scanning Fourier ptychographic system can also digitallypropagate the complex image to other planes, for example, to generate athree-dimensional image.

Although this aperture-scanning Fourier ptychographic imaging system isdescribed as configured to generate an aperture at an intermediateplane, this system in another case could generate a plurality ofapertures at the intermediate plane.

Aperture scanning Fourier ptychographic methods performed by aperturescanning Fourier ptychographic systems described herein comprise anacquisition process, a recovery process, and an optional displayprocess. During the acquisition process, the aperture scanner generatesan aperture at a plurality of N aperture locations and the radiationdetector captures at M sample times (t_(i), i=1 to M) a plurality of Mintensity images corresponding to different aperture locations. Duringthe recovery process, one or more complex images are determined usingthe plurality of M intensity images. During the optional displayprocess, the recovered complex images and other output is provided on adisplay. In some cases, M=N.

FIG. 7 is a schematic diagram of components of an aperture scanningFourier ptychographic system 17, according to embodiments. The aperturescanning Fourier ptychographic system 17 comprises an aperture scanningFourier ptychographic device 700 and optionally one or more componentsof a computing device 800. The aperture scanning Fourier ptychographicdevice 700 comprises a first optical system 730 (e.g., first objectivelens) configured to receive light from the sample 720, an aperturescanner 740 configured to generate an aperture at a plurality of Naperture locations in an intermediate plane (e.g., Fourier plane ofsample 720), a second optical system 750 (e.g., second objective lens)for receiving light through the aperture, and a detector 760 forcapturing M intensity images based on incident light from the secondoptical system 750.

The aperture scanning Fourier ptychographic device 700 further comprisesan optional (denoted by dotted line) illumination source 710 configuredto provide illumination to a sample 720. In this illustration, thesample 720 is provided to the aperture scanning Fourier ptychographicdevice 700 during an acquisition process as denoted by the dotted line.In other cases, the sample 720 is not included. In some cases, theillumination source 710 may provide a single coherent illumination beamfrom any direction. The computing device 800 comprises a processor 810(e.g., a microprocessor), a computer readable medium (CRM) 820 inelectrical communication with the processor 810, and a display 830 inelectrical communication with the processor 810. The processor 810 ofthe computing device 800 is also in electrical communication with thedetector 760 of the aperture scanning Fourier ptychographic device 700.In certain cases, the processor 810 may also be in electricalcommunication with the aperture scanner 740. In one case, for example,the processor 810 may be in electrically communication with the aperturescanner 740 and the light detector 760 to synchronize aperturegeneration with image acquisition. The computing device 800 can be invarious forms such as, for example, a smartphone, laptop, desktop,tablet, etc. Various forms of computing devices would be contemplated byone skilled in the art.

During a measurement process, the aperture scanner 740 generates anaperture at a plurality of N aperture locations, (X_(i), Y_(j)), i=1 tom, j=1 to n, in a plane (e.g., Fourier plane of the opticalarrangement). The first optical system 730 receives incident lightpropagating from the surface of the sample 720. The second opticalsystem 750 receives light as modulated by the aperture. The detector 760receives and measures the intensity distribution of light propagated bythe second optical system. The detector 760 captures or acquires anintensity distribution I_(i,j), i=1 to o, j=1 top at M (=o×p) sampletimes, t_(i=1 to M), to capture a plurality of M intensity images of thesample 720. In one aspect, each of the M intensity images corresponds toa different aperture location of the plurality of N aperture locations.

In certain aspects, one or more of the full field-of-view intensityimages captured by an aperture scanning Fourier ptychographic systemdescribed herein may be divided into one or more tile images. In thesecases, the processor may construct a higher resolution complex image foreach tile independently, and then combine the tile images to generatethe full field-of-view image. This ability to process tile imagesindependently allows for parallel computing. In these aspects, each tilemay be represented by a two-dimensional area. In polar spatialcoordinates, each tile may be a circular area or an oval area. Inrectilinear spatial coordinates, the full field-of view low resolutionimage may be divided up into a two-dimensional matrix of tiles in arectangular area. In some embodiments, the dimensions of atwo-dimensional square matrix of tiles may be in powers of two whenexpressed in number of pixels of the radiation sensor such as, forexample, a 256 by 256 matrix, a 64×64 matrix, etc.

In FIG. 7, the processor 810 is in electronic communication withdetector 760 to receive signal(s) with the image data corresponding to Mintensity images. The image data may include, for example, intensitydistributions, associated acquisition times, etc. During a recoveryprocess, the processor 810 can iteratively “stitch” together theplurality of M intensity images in Fourier space to recover awide-field, complex image of the sample 720 at the sample plane. Incertain aspects, the processor 810 can also digitally refocus thecomplex image to accommodate for any defocus of the sample and/oraberrations in the system. In certain aspects, the processor 810 canalso propagate the complex image to one or more planes. The image datafrom these propagated complex images at different planes can be used togenerate a three-dimensional image. In certain aspects, the processor810 can also generate a complex image at different illuminationwavelengths (RGB) to generate a complex color image.

The processor 810 is in electronic communication with CRM 820 (e.g.,memory) to communicate signals with image data to store/to/from the CRM820. Processor 810 is shown in electronic communication with display 830to be able to send image data and instructions to display thewide-field, complex image of the sample and other output, for example,to a user of the aperture scanning Fourier ptychographic system 17. Asused herein, electronic communication between components of aperturescanning Fourier ptychographic system 17 may be in wired or wirelessform.

The processor 810 (e.g., microprocessor) may also execute instructionsstored on the CRM 820 to perform one or more functions of aperturescanning Fourier ptychographic system. For example, the processor 810may execute instructions to perform one or more steps of the recoveryprocess of the aperture scanning Fourier ptychographic method. Asanother example, the processor 810 may execute instructions forgenerating an aperture at the plurality of aperture locations. Asanother example, the processor 810 may execute instructions stored onthe CRM 820 to perform one or more other functions of the aperturescanning Fourier ptychographic system such as, for example, 1)interpreting image data from the plurality of intensity images, 2)generating a higher resolution complex image from the image data, and 3)displaying one or more images or other output from the aperture scanningFourier ptychographic method on the display 830.

The CRM (e.g., memory) 820 can store instructions for performing some ofthe functions of the aperture scanning Fourier ptychographic system. Theinstructions are executable by the processor 810 or other processingcomponents of the aperture scanning Fourier ptychographic system. TheCRM 820 can also store the (lower resolution) intensity and higherresolution complex images, and other data produced by the aperturescanning Fourier ptychographic system.

The aperture scanning Fourier ptychographic system also includes adisplay 830 in electronic communication with the processor 810 toreceive data (e.g., image data) and provide output data (e.g., images)to an operator of the aperture scanning Fourier ptychographic system.The image display 830 may be a color display or a black and whitedisplay. In addition, the display 830 may be a two-dimensional displayor a three-dimensional display. In one embodiment, the display 830 maybe capable of displaying multiple views.

Modifications, additions, or omissions may be made to the aperturescanning Fourier ptychographic system 17 or aperture scanning Fourierptychographic device 700 without departing from the scope of thedisclosure. In addition, the components of aperture scanning Fourierptychographic system 17 or the aperture scanning Fourier ptychographicdevice 700 may be integrated or separated according to particular needs.For example, the computing device 800 or components thereof may beintegrated into the aperture scanning Fourier ptychographic device 700.In some embodiments, the processor 810 or other suitable processor maybe part of the aperture scanning Fourier ptychographic device 700. Insome cases, the processor 810 may be integrated into the radiationdetector 760 so that the radiation detector 760 performs the functionsof the processor 810. As another example, the CRM 820 and/or display 830may be omitted from the aperture scanning Fourier ptychographic system17 in certain cases.

For simplicity, the first and second optical systems (e.g. first andsecond lenses) of certain aperture-scanning Fourier ptychographicimaging systems herein are described having the same focal length, f, ina 4f optical arrangement. It will be understood that the first opticalsystem can have a different focal length than the second optical system.For example, the first optical system may have a first focal length off₁ that is different that the second focal length f₂ of the secondoptical system. In this case, the sample plane is located at a distanceof first focal length f₁ from the first optical system, the detectorplane will be at a distance of the second focal length f₂ from thesecond optical system, and the Fourier plane will be at a distance of f₁from the first optical system and a distance of f₂ from the secondoptical system.

In many aspects described herein, the aperture can be generated at aplurality of N aperture locations in a Fourier plane of the sample.However, it would be understood that the aperture could be generated inanother intermediate plane conjugate to the sample such as, for example,the aperture plane of a compound lens system or the back-focal plane ofa microscope objective.

In certain aspects, an aperture scanning Fourier ptychographic systemmay further comprise a receptacle for receiving the sample at a samplesurface. The sample surface may be part of a component of or a separatecomponent of the aperture scanning Fourier ptychographic system.

IV. Aperture-Scanning Fourier Ptychographic Imaging Methods

In certain aspects, an aperture scanning Fourier ptychographic methodcomprises an acquisition process, a recovery process, and an optionaldisplay process. In the acquisition process, a plurality of M intensitylower resolution images are acquired, each intensity image correspondingto a different aperture location at the intermediate plane of theaperture scanning Fourier ptychographic system. Each intensity image isbased on an intensity (amplitude) distribution measured at the detectorplane at a particular acquisition time, t_(i=1toM). The light detectormeasures incident light received from the second optical system, whichreceives light from the aperture.

In FIGS. 8, 9A and 9B and their associated description, subscript “h”refers to higher resolution, complex image, subscript “l” refers tolower resolution intensity, subscript “f” refers to focused position,subscript “m” refers to measured, and subscript “s” refers to sampled.

FIG. 8 is a flowchart of an aperture scanning Fourier ptychographicmethod performed by an aperture scanning Fourier ptychographic system.The aperture scanning Fourier ptychographic method comprises anacquisition process (steps 1100, 1200, and 1300), a recovery process(steps 1400 and 1500), an optional propagation step and an optionaldisplay process (step 1600).

At step 1100, the illumination source provides illumination to a sampleduring M sample times t_(i=1 . . . M). The first optical system receivesincident light from the sample. In certain cases, the illuminationsource may provide illumination of different wavelengths at differentsample times. For example, the illumination source may provide RGBillumination of three wavelengths λ₁, λ₂, and λ₃ corresponding to red,green, blue colors, respectively, for a color imaging implementation. Inluminescence imaging examples, the illumination source may provideillumination that is of wavelength(s) for exciting fluorophore in thesample. In these examples, the illumination source may be located anddirected to provide illumination directed away from the next element inthe optical arrangement. For example, the illumination source may bedirected away from the first optical system.

At step 1200, an aperture scanner generates an aperture (or plurality ofapertures) at a plurality of N aperture locations, (X_(i), Y_(j)), i=1to m, j=1 to n, in an intermediate (e.g., Fourier) plane of the opticalarrangement. The aperture scanner may generate the aperture at thedifferent locations based on instructs that define the order of theaperture locations. These instructions may be implemented with aprocessor and may be stored on computer readable medium. The wave vectorin x and y directions can be denoted as k_(xi), and k_(yi). The secondoptical system may receive incident light as modulated by the aperture.In some cases, the neighboring apertures in the plurality of aperturelocations have an overlapping region.

The detector receives and measures the intensity distribution of lightpropagated by a second optical system receiving incident light from theaperture. At step 1300, the radiation detector acquires a snapshotintensity distribution measurement at each of the M sample times,t_(i=1toM) to acquire a plurality of M intensity images I_(i,j), i=1 too, j=1 top where M=o×p. Each of the M intensity images acquired by thelight detector corresponds to a different aperture location of theplurality of N aperture locations. Each of the M intensity imagesacquired by the light detector is also associated with a region inFourier space. In certain aspects, there are overlapping areas betweenneighboring regions in Fourier space. In one embodiment, there is anoverlapping area between neighboring regions of 2% to 99.5% of the areaof one of the regions. In another embodiment, there is an overlappingarea between neighboring regions of 65% to 75% of the area of one of theregions. In one embodiment, there is an overlapping area betweenneighboring regions of about 65% of the area of one of the regions.

At steps 1400 and 1500, a higher (i.e. improved) resolution, compleximage of the sample is recovered based on the plurality of M intensitydistribution measurements, I_(i,j), i=1 to o, j=1 captured at step 1300.

At step 1400, a higher resolution complex image: √{square root over(I_(h))}e^(iφ) ^(h) is initialized in the spatial domain, and a Fouriertransform is applied to the initial value to obtain an initializedFourier transformed image Ĩ_(h). The initialized higher-resolutionsolution may be an initial guess. In some cases, the initial guess maybe determined as a random complex matrix (for both intensity and phase).In other cases, the initial guess may be determined as an interpolationof the intensity distribution measurement with a random phase. Anexample of an initial guess is φ=0 and I_(h) interpolated from anyintensity image of the sample area. Another example of an initial guessis a constant value. The Fourier transform of the initial guess can be abroad spectrum in the Fourier domain.

At step 1500, the higher-resolution image of the sample area iscomputationally constructed by iteratively combining intensitymeasurements in Fourier space using a processor, which may be part of ora separate component of the of the an aperture scanning Fourierptychographic system.

At an optional step 1600, a display may receive image data such as ahigher resolution complex image data and/or other data from theprocessor, and display the data on the display.

Although an aperture scanning Fourier ptychographic system may notdirectly measure phase information, the aperture scanning Fourierptychographic system may determine this data during its recoveryprocess. The phase data can be used to generate a complex image of thesample. In addition, certain aperture scanning Fourier ptychographicmethods can use phase information for aberration correction. Forexample, certain aperture scanning Fourier ptychographic methodsintroduces a phase map to the coherent optical transfer function tocompensate for aberrations at the pupil plane during the iterative imagerecovery process. Examples of image recovery processes are describedwith reference to FIGS. 9A and 9B discussed in the following sections.

A) Digital Re-Focusing and Wavefront Correction

Consider a situation where a sample being imaged is illuminated by alight field. The optical transmission exiting the sample surfaceincludes both amplitude and phase spatial variations. In a conventionalbright field microscope, this light field is collected and refocused toform an image of the sample at the image plane. Conventional lightsensors and the human eye can only detect amplitude (intensity)variations, but not the phase variations.

There are advantages to imaging platforms that can collect both theamplitude and phase variations at the image plane and connect that databack to the optical transmission exiting the sample surface. Forexample, this set of amplitude and phase data can be used to performcomputational refocusing, which allows for imaging at any given planebelow the sample's surface. As another example, this set of amplitudeand phase data can be used to correct for optical aberrations in opticalimaging systems. Optical aberrations present physical limitations thatmay prevent certain imaging systems from performing at their theoreticalresolution dictated by general optical principles. For example, a camerawith a 50 mm lens (e.g., a Nikon Nikkor 50 mm f/1.2) having a field ofview of over 1 cm diameter and a numerical aperture (NA) of about 0.4,should theoretically be capable of imaging with sub-micron opticalresolution, but optical aberrations limit it to 10's microns resolution.

FIG. 9A is a flowchart illustrating an example of sub-steps, one or moreof which may be included in step 1500 of the aperture scanning Fourierptychographic method of FIG. 8. One or more of these steps may beperformed by a processor (e.g., processor 810) of the aperture scanningFourier ptychographic system. The illustrated flowchart includesoptional digital wavefront correction steps 1605 and 1645. Step 1605provides a connection between the actual sample profile and the capturedintensity data (which may include aberrations) with multiplication of apupil function: e^(iφ(k) ^(x) ^(,k) ^(y) ⁾. Step 1645 inverts thisconnection to determine an aberration-free reconstructed complex imageof the sample.

Sample defocus can be implemented by introducing the defocus phasefactor to the pupil plane (i.e., a defocus aberration):

$\begin{matrix}{{^{\; {\phi {({k_{x},k_{y}})}}} = ^{{\sqrt{{({2{\pi/\lambda}})}^{2} - k_{x}^{2} - k_{y}^{2}} \cdot z_{0}}}},{{k_{x}^{2} + k_{y}^{2}} < \left( {{{NA} \cdot 2}{\pi/\lambda}} \right)^{2}}} & \left( {{Eqn}.\mspace{14mu} 4} \right)\end{matrix}$

where k_(x) and k_(y) are the wavenumbers at the pupil plane, z₀ is thedefocus distance, and NA is the numerical aperture of an optical element(e.g., first optical system and/or second optical system).

At step 1605, the initial complex, higher resolution Fourier transformedimage Ĩ_(h) is multiplied by a phase factor e^(iφ(k) ^(x) ^(,k) ^(y) ⁾or exp(iφ(k_(x),k_(y))) in the Fourier domain.

At step 1610, low-pass filtering of the higher-resolution image √{squareroot over (I_(h))}e^(iφ) ^(h) in the Fourier domain is performed togenerate a lower resolution intensity image √{square root over(I_(l))}e^(iφ) ^(l) or √{square root over (I_(l))} exp(iφ_(l)) for anaperture location associated with a wave vector (k_(x) ^(i), k_(y)^(i)). The Fourier transform of the higher-resolution image is Ĩ_(h) andthe Fourier transform of the lower resolution intensity image for aparticular aperture location is Ĩ_(l). In the Fourier domain, theaperture scanning Fourier ptychographic method filters the low-passregion from the spectrum Ĩ_(h) of the higher-resolution image √{squareroot over (I_(h))}e^(iφ) ^(h) . In some cases, this low-pass region maybe a circular aperture with a radius of NA*k₀, where k₀ equals 2π/λ (thewave number in vacuum), given by the coherent optical transfer functionof an optical system (e.g., first optical system and/or second opticalsystem). In Fourier space, the location of the low-pass regioncorresponds to a particular aperture location in the spatial domain.

At step 1630, the computed amplitude component √{square root over(I_(lf))} of the intensity image at the in-focus plane, √{square rootover (I_(lf))}e^(iφ) ^(lf) , is replaced with the square root of theintensity measurement √{square root over (I_(lfm))} measured by thelight detector of the aperture scanning Fourier ptychographic system.This forms an updated lower resolution target: √{square root over(I_(lfm))}e^(iφ) ^(lf) .

At step 1645, the updated lower resolution target: √{square root over(I_(lfm))}e^(iφ) ^(lf) is multiplied by an inverse phase factore^(−iφ(k) ^(x) ^(,k) ^(y) ⁾ or exp(−1iφ(k_(x),k_(y))) in Fourier domain.

At step 1650, a Fourier transform is applied to the updated target imagepropagated to the sample plane: √{square root over (I_(ls))}e^(iφ) ^(ls), and this data is updated in the corresponding region ofhigher-resolution solution √{square root over (I_(h))}e^(iφ) ^(h) in theFourier space corresponding to the corresponding to the incidence wavevector (k_(x) ^(i), k_(y) ^(i)).

At step 1660, it is determined whether steps 1605 through 1650 have beencompleted for all aperture N locations. If steps 1605 through 1650 havenot been completed for all aperture N locations, steps 1605 through 1650are repeated for the next aperture location.

In most embodiments, the neighboring regions in Fourier space, which areiteratively updated for each aperture location, overlap each other. Inthe overlapping area between updated overlapping regions, the aperturescanning Fourier ptychographic system has multiple samplings over thesame Fourier space. The aperture locations determine the area of theoverlapping area. In one embodiment, the overlapping area betweenneighboring regions may have an area that is between 2% to 99.5% of thearea of one of the neighboring regions. In another embodiment, theoverlapping area between neighboring regions may have an area that isbetween 65% to 75% of the area of one of the neighboring regions. Inanother embodiment, the overlapping area between neighboring regions mayhave an area that is about 65% of the area of one of the neighboringregions. In certain embodiments, each overlapping region has the samearea.

At step 1670, it is determined whether the solution for thehigher-resolution image has converged. For example, convergence may bedetermined if the higher-resolution complex image is a self-consistentsolution. In one case, the previous higher-resolution complex image ofthe previous iteration or initial guess is compared to the presenthigher-resolution solution, and if the difference is less than a certainvalue, the solution may have converged to a self-consistent solution. Ifit is determined that the solution has not converged, then steps 1605through 1670 are repeated. In one embodiment, steps 1605 through 1670are repeated once. In other embodiments, steps 1605 through 1670 arerepeated twice or more. If the solution has converged, the convergedsolution in Fourier space is transformed to the spatial domain torecover a higher-resolution image √{square root over (I_(h))}e^(iφ) ^(h). If it is determined that the solution has converged at step 1570, thenthe method may proceed to optional step 1600, the method may end, orother optional additional step(s) may be performed such as additionaldefocus or aberration correction steps.

If the defocus distance is unknown, the aperture scanning Fourierptychographic method can digitally adjust the ‘z’ parameter to differentvalues based on a computation of the auto-focusing index from Eqn. 4.The aperture scanning Fourier ptychographic method can then constructthe corresponding images, and select the sharpest image. This approachcan also be extended to image a tiled sample. In this case, the aperturescanning Fourier ptychographic method can digitally adjust the ‘z’parameter to achieve acuity for each tiled region of the whole image andcombine the in-focus regions to form a fully focused image of the fullfield of view.

In other embodiments, alternative digital multiplicative phase factorscan be included in multiplication steps 1605 and 1645 to correct for avariety of aberrations, as long as the factors correctly model theemployed optics.

A limitation of conventional high-NA microscopes is a limited depth-offield. As an example, the depth-of-field of a conventional microscopewith a 20× objective lens with 0.4 NA is about 5 μm. With a conventionalmicroscope, resolution degrades as the sample moves away from thein-focus plane due to its limited depth-of-field. To achieve optimalresolution using a conventional microscope, the operator typically needsto move the stage to mechanically bring the sample back into focus. Inthis regard, a precise mechanical stage is needed in the conventionalmicroscope to bring a sample into the in-focus position with sub-micronaccuracy.

In certain embodiments, an aperture scanning Fourier ptychographicsystem implements an aperture scanning Fourier ptychographic method inwhich a sample can be refocused digitally rather than mechanically. Inthese cases, the aperture scanning Fourier ptychographic methodcomputationally refocuses the out-of-focus sample during the recoveryprocess. Using digital refocusing, the aperture scanning Fourierptychographic system can expand its depth-of focus beyond the physicallimitations of its optical element.

During operation of an aperture scanning Fourier ptychographic system,the z-position of the sample may not be known a priori. In certainaspects, an aperture scanning Fourier ptychographic method may include adigital auto-focusing step that determines the z-position of the sampleand uses this z-position to digitally refocus. For example, the aperturescanning Fourier ptychographic method of FIG. 8 may further comprise astep during or before step 1520 that computes the z-position of thesample. The aperture scanning Fourier ptychographic system may performdigital autofocusing by using a processor to perform steps 1520 and 1540in FIG. 8 using the computed z-position of the sample. To compute thez-position of the sample, the aperture scanning Fourier ptychographicmethod determines an auto-focusing index parameter. The auto-focusingindex can be defined by the following equation:

Auto-focusing index:1/Σabs(√{square root over (I _(lf))}−√{square rootover (I _(lfm))})  (Eqn. 2)

-   -   Where: √{square root over (I_(lf))} is the amplitude image from        the low-pass filtering, and √{square root over (I_(lfm))} is the        actual intensity measurement

The summation in Eqn. 2 is for all aperture locations. After theaperture scanning Fourier ptychographic method computes the estimatedz-position of the sample, the aperture scanning Fourier ptychographicmethod can digitally refocus to the estimated z-position. In some cases,the recovered solution of the higher-resolution image has been found toconverge more quickly when using an accurate z-position.

B) Another Example of a Recovery Process

FIG. 9B is a flowchart describes an example of alternate sub-steps ofstep 1500 of FIG. 8. In this case, step 1500 comprises step 1510, step1530, step 1550, step 1560, step 1570, step 1580, and step 1590. Step1500 may optionally comprise steps 1520 and 1540. Optional steps 1520and 1540 may be performed if the sample is out-of-focus by the amount ofz₀. One or more of the sub-steps in FIG. 9B can be performed by aprocessor.

At step 1510, low-pass filtering of the higher-resolution image √{squareroot over (I_(h))}e^(iφ) ^(h) in the Fourier domain is performed togenerate a low-resolution image √{square root over (I_(l))}e^(iφ) ^(l)for a particular aperture location associated with a wave vector (k_(x)^(i), k_(y) ^(i)). The Fourier transform of the higher-resolution imageis Í_(h) and the Fourier transform of the low-resolution image for aparticular aperture location is Ĩ_(l). In the Fourier domain, theaperture scanning Fourier ptychographic method filters a low-pass regionfrom the spectrum Ĩ_(h) of the higher-resolution image √{square rootover (I_(h))}e^(iφ) ^(h) . In cases with an optical element in the formof an objective lens, this region may be a circular aperture with aradius of NA*k₀, where k₀ equals 2π/λ (the wave number in vacuum), givenby the coherent transfer function of an objective lens. In Fourierspace, the location of the low-pass region corresponds to the aperturelocation. The region may be centered about a position (−k_(x)^(i),−k_(y) ^(i)) in the Fourier domain of √{square root over(I_(h))}e^(iφ) ^(h) .

At optional step 1520, the low-resolution image, √{square root over(I_(l))}e^(iφ) ^(l) is propagated in the Fourier domain to the in-focusplane at z=0 of the optical element to determine the low-resolutionimage at the focused position: √{square root over (I_(lf))}e^(iφ) ^(lf). In one embodiment, Step 1520 can be performed by Fourier transformingthe low-resolution image √{square root over (I_(l))}e^(iφ) ^(l) ,multiplying by a phase factor in the Fourier domain, and inverse Fouriertransforming to obtain √{square root over (I_(lf))}e^(iφ) ^(lf) . Inanother embodiment, step 1520 can be performed by the mathematicallyequivalent operation of convolving the low-resolution image √{squareroot over (I_(l))}e^(iφ) ^(l) with the point-spread-function for thedefocus. In another embodiment, step 1520 can be performed as anoptional sub-step of step 1510 by multiplying by multiplying Ĩ_(l) by aphase factor in the Fourier domain before performing the inverse Fouriertransform to produce √{square root over (I_(lf))}e^(iφ) ^(lf) . Optionalstep 1520 need not be included if the sample is located at the in-focusplane (z=0) of the optical element.

At step 1530, the computed amplitude component √{square root over(I_(lf))} of the low-resolution image at the in-focus plane, √{squareroot over (I_(lf))}e^(iφ) ^(lf) , is replaced with the square root ofthe low-resolution intensity measurement √{square root over (I_(lfm))}measured by the radiation detector of the aperture scanning Fourierptychographic system. This forms an updated low resolution target:√{square root over (I_(lfm))}e^(iφ) ^(lf) .

At optional step 1540, the updated low-resolution image √{square rootover (I_(lfm))}e^(iφ) ^(lf) may be back-propagated to the sample plane(z=z₀) to determine √{square root over (I_(ls))}e^(iφ) ^(ls) . Optionalstep 1540 need not be included if the sample is located at the in-focusplane of the optical element, that is, where z₀=0. In one case, step1540 can be performed by taking the Fourier transform of the updatedlow-resolution image √{square root over (I_(lfm))}e^(iφ) ^(lf) andmultiplying in the Fourier space by a phase factor, and then inverseFourier transforming it. In another case, step 1540 can be performed byconvolving the updated low-resolution image √{square root over(I_(lfm))}e^(iφ) ^(lf) with the point-spread-function of the defocus. Inanother case, step 1540 can be performed as a sub-step of step 1550 bymultiplying by a phase factor after performing the Fourier transformonto the updated target image.

At step 1550, a Fourier transform is applied to the updated target imagepropagated to the sample plane: √{square root over (I_(ls))}e^(iφ) ^(ls), and this data is updated in the corresponding region ofhigher-resolution solution √{square root over (I_(h))}e^(iφ) ^(h) in theFourier space corresponding to the corresponding to the incidence wavevector (k_(x) ^(i), k_(y) ^(i)) and associate aperture location.

At step 1560, it is determined whether steps 1510 through 1560 have beencompleted for all N aperture locations. If steps 1510 through 1560 havenot been completed for all N aperture locations, steps 1510 through 1560are repeated for the next aperture location.

In most embodiments, the neighboring regions in Fourier space, which areiteratively updated for each aperture location, overlap each other. Inthe overlapping area between updated overlapping regions, the aperturescanning Fourier ptychographic method system has multiple samplings overthe same Fourier space. The aperture locations determine the area of theoverlapping area. In one embodiment, the overlapping area betweenneighboring regions may have an area that is between 2% to 99.5% of thearea of one of the neighboring regions. In another embodiment, theoverlapping area between neighboring regions may have an area that isbetween 65% to 75% of the area of one of the neighboring regions. Inanother embodiment, the overlapping area between neighboring regions mayhave an area that is about 65% of the area of one of the neighboringregions. In certain embodiments, each overlapping region has the samearea.

At step 1570, it is determined whether the solution for thehigher-resolution image has converged. For example, convergence may bedetermined if the higher-resolution complex image is a self-consistentsolution. In one case, the previous higher-resolution complex image ofthe previous iteration or initial guess is compared to the presenthigher-resolution solution, and if the difference is less than a certainvalue, the solution may have converged to a self-consistent solution. Ifit is determined that the solution has not converged, then steps 1510through 1570 are repeated. In one embodiment, steps 1510 through 1560are repeated once. In other embodiments, steps 1510 through 1560 arerepeated twice or more. If the solution has converged, the processortransforms the converged solution in Fourier space to the spatial domainto recover a higher-resolution image √{square root over (I_(h))}e^(iφ)^(h) . If the processor determines that the solution has converged atstep 1570, then the process may proceed to optional step 1600, themethod may end, or other optional additional step(s) may be performedsuch as additional defocus or aberration correction steps.

C) Tile Imaging

In some embodiments, an aperture scanning Fourier ptychographic methodmay include a tile imaging process that divides the captured intensityimages into a plurality of intensity tile images, independently acquiresa higher-resolution image for each of the tiles, and then combines thehigher-resolution tile images to generate a full field-of-viewhigher-resolution image. In some cases, the higher-resolution tileimages may be combined with an image blending process. An example of animage blending process is alpha blending which can be found in PCTpublication WO1999053469, entitled “A system and method for performingblending using an over sampled buffer,” filed on Apr. 7, 1999, which ishereby incorporated by reference in its entirety for this example. Sincethe higher-resolution images of the tiles may be acquired independently,this aperture scanning Fourier ptychographic method may allow forparallel computing, which may reduce computation time, and may alsoreduce memory requirements. Moreover, the light from each light elementmay be accurately treated as a plane wave for each tile. The incidentwavevector for each tile can be expressed as:

$\begin{matrix}{\left( {k_{x}^{i},k_{y}^{i}} \right) = {\frac{2\pi}{\lambda}\begin{pmatrix}{\frac{\left( {x_{c} - x_{i}} \right)}{\sqrt{\left( {x_{c} - x_{i}} \right)^{2} + \left( {y_{c} - y_{i}} \right)^{2} + h^{2}}},} \\\frac{\left( {y_{c} - y_{i}} \right)}{\sqrt{\left( {x_{c} - x_{i}} \right)^{2} + \left( {y_{c} - y_{i}} \right)^{2} + h^{2}}}\end{pmatrix}}} & \left( {{Eqn}.\mspace{14mu} 1} \right)\end{matrix}$

where (x_(c),y_(c)) is the central position of each tile of the fullfield-of-view intensity image, (x_(i),y_(i)) is the position of thei^(th) light element, and h is the distance between the illuminator andthe sample. Furthermore, this aperture scanning Fourier ptychographicmethod can assign a specific aberration-correcting pupil function toeach tile in some cases.

FIG. 10 is a flowchart of an aperture scanning Fourier ptychographicmethod with tile imaging, according to certain aspects. This aperturescanning Fourier ptychographic method can be performed by an aperturescanning Fourier ptychographic system. To take advantage of parallelprocessing capabilities, the aperture scanning Fourier ptychographicsystem used to perform the method comprises a processor with parallelprocessing capabilities such as, for example, the GPU unit or aprocessor having multiple cores (i.e. independent central processingunits). In this example, the aperture scanning Fourier ptychographicmethod comprises a measurement process (steps 1101, 1201, and 1301), arecovery process (steps 1351, 2401 (i-M), 2501(i-M), 2591), and anoptional display process (step 1601). The measurements process (steps1101, 1201, and 1301) and display process (step 1600) are similar tothose steps described with reference to FIG. 8.

At step 1351, the processor divides the full field-of-view into aplurality of tiles such as, for example, a two-dimensional matrix oftiles. The dimensions of a two-dimensional square matrix of tiles may bein powers of two such as, for example, a 256 by 256 matrix, a 64×64matrix, etc. In one example, the processor may divide up a full field ofview of 5,280×4,380 pixels into tiles having an area of 150×150 pixels.

Next, the processor initializes the higher-resolution image: √{squareroot over (I_(h))}e^(iφ) ^(h) in the spatial domain for each tile (1 toM) independently using parallel computing (step 2400(1) . . . step2400(M)). A Fourier transform is applied to the initial guess. In somecases, the initial guess may be determined as a random complex matrix(for both intensity and phase). In other cases, the initial guess may bedetermined as an interpolation of the intensity measurement with arandom phase. An example of an initial guess is φ=0 and I_(hr) of anyintensity image of the sample area. Another example of an initial guessis a constant value. The Fourier transform of the initial guess can be abroad spectrum in the Fourier domain.

At step 2501(1) . . . step 2501(M), the processor computationallyconstructs a higher-resolution image of each tile (1 to M) independentlyusing parallel computing. The processor computationally constructs thehigher-resolution image of each tile by iteratively combining intensityimages in Fourier space. Steps 1521 and 1541 may be included if sampleout of focus.

At step 2591, the processor combines the higher-resolution tile imagesinto a full field-of view higher-resolution image. In some cases,combining tile images comprises an imaging-blending process such as, forexample, alpha blending.

Color imaging capability is pivotal in pathology and histology. Incertain embodiments, an aperture scanning Fourier ptychographic system10 capable of color imaging comprises an illumination source that canprovide red, green, and blue illuminations. The aperture scanningFourier ptychographic method combines the higher-resolution imageresults from red, green, and blue LED illumination into eachcorresponding color channel to form a final higher-resolution colorimage. Three images are generated corresponding to red, green, and blue,which are combined to form a higher resolution color image.

VI. Subsystems

FIG. 11 is a block diagram of subsystems that may be present in certainaperture scanning Fourier ptychographic systems described herein. Forexample, an aperture scanning Fourier ptychographic system may include aprocessor. The processor may be a component of the aperture scanningFourier ptychographic system in some cases. The processor may be acomponent of the radiation detector in some cases.

The various components previously described in the Figures may operateusing one or more of the subsystems to facilitate the functionsdescribed herein. Any of the components in the Figures may use anysuitable number of subsystems to facilitate the functions describedherein. Examples of such subsystems and/or components are shown in aFIG. 11. The subsystems shown in FIG. 11 are interconnected via a systembus 2425. Additional subsystems such as a printer 2430, keyboard 2432,fixed disk 2434 (or other memory comprising computer readable media),display 830, which is coupled to display adapter 2438, and others areshown. Peripherals and input/output (I/O) devices, which couple to I/Ocontroller 2440, can be connected by any number of means known in theart, such as serial port 2442. For example, serial port 2442 or externalinterface 2444 can be used to connect the computing device 200 to a widearea network such as the Internet, a mouse input device, or a scanner.The interconnection via system bus 2425 allows the processor tocommunicate with each subsystem and to control the execution ofinstructions from system memory 2446 or the fixed disk 2434, as well asthe exchange of information between subsystems. The system memory 2446and/or the fixed disk 2434 may embody the CRM 220 in some cases. Any ofthese elements may be present in the previously described features.

In some embodiments, an output device such as the printer 2430 ordisplay 830 of the aperture scanning Fourier ptychographic system canoutput various forms of data. For example, the aperture scanning Fourierptychographic system can output 2D color/monochromatic images (intensityand/or phase), data associated with these images, or other dataassociated with analyses performed by the aperture scanning Fourierptychographic system.

Modifications, additions, or omissions may be made to any of theabove-described embodiments without departing from the scope of thedisclosure. Any of the embodiments described above may include more,fewer, or other features without departing from the scope of thedisclosure. Additionally, the steps of the described features may beperformed in any suitable order without departing from the scope of thedisclosure.

It should be understood that the present invention as described abovecan be implemented in the form of control logic using computer softwarein a modular or integrated manner. Based on the disclosure and teachingsprovided herein, a person of ordinary skill in the art will know andappreciate other ways and/or methods to implement the present inventionusing hardware and a combination of hardware and software.

Any of the software components or functions described in thisapplication, may be implemented as software code to be executed by aprocessor using any suitable computer language such as, for example,Java, C++ or Perl using, for example, conventional or object-orientedtechniques. The software code may be stored as a series of instructions,or commands on a CRM, such as a random access memory (RAM), a read onlymemory (ROM), a magnetic medium such as a hard-drive or a floppy disk,or an optical medium such as a CD-ROM. Any such CRM may reside on orwithin a single computational apparatus, and may be present on or withindifferent computational apparatuses within a system or network.

Although the foregoing disclosed embodiments have been described in somedetail to facilitate understanding, the described embodiments are to beconsidered illustrative and not limiting. It will be apparent to one ofordinary skill in the art that certain changes and modifications can bepracticed within the scope of the appended claims.

One or more features from any embodiment may be combined with one ormore features of any other embodiment without departing from the scopeof the disclosure. Further, modifications, additions, or omissions maybe made to any embodiment without departing from the scope of thedisclosure. The components of any embodiment may be integrated orseparated according to particular needs without departing from the scopeof the disclosure.

What is claimed is:
 1. An aperture-scanning Fourier ptychographicimaging device, comprising: a first optical element configured toreceive light from a sample; a second optical element; an aperturescanner configured to generate an aperture at a plurality of aperturelocations in an intermediate plane, the aperture configured to passincident light at the aperture from the first optical element to thesecond optical element; a light detector configured to receive lightfrom the second optical element and to acquire a plurality of intensityimages associated with different aperture locations; and a processorconfigured to construct a complex image of the sample by iterativelyupdating regions in Fourier space with the acquired intensity images. 2.The aperture-scanning Fourier ptychographic imaging device of claim 1,wherein the intermediate plane is a Fourier plane associated with asample plane.
 3. The aperture-scanning Fourier ptychographic imagingdevice of claim 1, wherein each of the plurality of intensity imagesacquired by the light detector uniquely corresponds to a differentaperture location of the plurality of aperture locations.
 4. Theaperture-scanning Fourier ptychographic imaging device of claim 1,wherein the aperture scanner is further configured to generateadditional apertures at the intermediate plane to form a plurality ofapertures during each acquisition time.
 5. The aperture-scanning Fourierptychographic imaging device of claim 1, further comprising an apertureoverlap between adjacent aperture locations in the plurality of aperturelocations.
 6. The aperture-scanning Fourier ptychographic imaging deviceof claim 5, wherein the overlap is at least about 70% of an area of theaperture.
 7. The aperture-scanning Fourier ptychographic imaging deviceof claim 5, wherein the overlap is at least about 75% of an area of theaperture.
 8. The aperture-scanning Fourier ptychographic imaging deviceof claim 5, wherein the overlap is between 20% and 90% of an area of theaperture.
 9. The aperture-scanning Fourier ptychographic imaging deviceof claim 1, wherein the first optical element and/or the second opticalelement is a lens.
 10. The aperture-scanning Fourier ptychographicimaging device of claim 1, wherein the first optical element and secondoptical element are in a 4f configuration.
 11. The aperture-scanningFourier ptychographic imaging device of claim 1, wherein the firstoptical element has a first focal length, and is located the first focallength from the sample plane, wherein the second optical element has asecond focal length, and is located the second focal length from theintermediate plane, and wherein the intermediate plane is a located afirst focal length away from the first optical element an is located thefirst focal length away from the first optical element.
 12. Theaperture-scanning Fourier ptychographic imaging device of claim 11,wherein the light detector is located at the second focal length fromthe second optical element.
 13. The aperture-scanning Fourierptychographic imaging device of claim 1, wherein the aperture scanner isa spatial light modulator configured to display the aperture as areflective element.
 14. The aperture-scanning Fourier ptychographicimaging device of claim 13, wherein the spatial light modulatorcomprises a liquid crystal on silicon display for displaying thereflective element.
 15. The aperture-scanning Fourier ptychographicimaging device of claim 1, wherein the aperture scanner comprises adigital micromirror device.
 16. The aperture-scanning Fourierptychographic imaging device of claim 15, wherein the aperture comprisesone or more micromirrors oriented at a first angle to reflect incidentlight to the second optical element, wherein an area surrounding theaperture comprises one or more micromirrors oriented at a second angleto reflect incident light away from the second optical element.
 17. Anaperture-scanning Fourier ptychographic imaging method, comprising:illuminating a sample; receiving incident light at a first opticalelement from the sample; generating an aperture at a plurality oflocations at an intermediate plane; passing incident light at theaperture from the first optical element to a second optical element;acquiring a plurality of intensity images using a detector receivinglight from the second optical element; and constructing a complex imageof the sample by iteratively updating regions in Fourier space with theplurality of intensity images.
 18. The aperture-scanning Fourierptychographic imaging method of claim 17, wherein the intermediate planeis a Fourier plane corresponding to a sample plane.
 19. Theaperture-scanning Fourier ptychographic imaging method of claim 17,further comprising generating additional apertures at the intermediateplane to form a plurality of apertures at the intermediate during eachacquisition time.
 20. The aperture-scanning Fourier ptychographicimaging method of claim 17, wherein generating the aperture comprisesdisplaying one or more reflective elements on a display of a spatiallight modulator.
 21. The aperture-scanning Fourier ptychographic imagingmethod of claim 17, wherein generating the aperture comprises orientingone or more micromirrors to reflect incident light to the second opticalelement.
 22. The aperture-scanning Fourier ptychographic imaging methodof claim 17, wherein the plurality of intensity images captured by thedetector uniquely correspond to different aperture locations of theplurality of aperture locations.
 23. The aperture-scanning Fourierptychographic imaging method of claim 17, wherein there is an apertureoverlap at adjacent aperture locations of the plurality of aperturelocations.
 24. The aperture-scanning Fourier ptychographic imagingmethod of claim 17, wherein the aperture overlap is at least about 70%of an area of the aperture.
 25. The aperture-scanning Fourierptychographic imaging method of claim 17, further comprising propagatingthe complex image to one or more planes.
 26. The aperture-scanningFourier ptychographic imaging method of claim 17, wherein constructing acomplex image of the sample by iteratively updating regions in Fourierspace with the plurality of intensity images, comprises: (a)initializing a current higher-resolution image in Fourier space; (b)filtering an overlapping region of the current higher-resolution imagein Fourier space to generate an intensity image for an aperture locationof the plurality of aperture locations; (c) replacing intensity of theintensity image with an intensity measurement; and (d) updating theoverlapping region in Fourier space with the intensity image withmeasured intensity.